2024-09-17T09:12:06.482 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T09:12:06.498 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T09:12:06.579 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908296 branch: reef description: orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} email: ceph-qa@ceph.com first_in_suite: false job_id: '7908296' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: true mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - MON_DOWN - mons down - mon down - out of quorum - CEPHADM_STRAY_DAEMON - CEPHADM_FAILED_DAEMON - CEPHADM_AGENT_DOWN log-only-match: - CEPHADM_ sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: reef sha1: 1cbdc9211df088cccbd94423c7cf3a2b85fe432b owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 5234 sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 sleep_before_teardown: 0 subset: 46/64 suite: orch suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 1cbdc9211df088cccbd94423c7cf3a2b85fe432b targets: smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNg6foFUA+2astYu4+zG5YD6PJE4hH6/vTrj4BPo/RR9gcy9EECPhiWltNDj7SlMgvdvfb1gwn+M5JXnaSctvdw= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIC59dp1IV3jtFCOoVJ4dWv/eoUa5cTy65J2KbidiVIQciti9qhtHulphxkKC8ko3x8kSBtphfnBTafuq9HkH2o= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_22:08:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T09:12:06.579 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa; will attempt to use it 2024-09-17T09:12:06.580 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_1cbdc9211df088cccbd94423c7cf3a2b85fe432b/qa/tasks 2024-09-17T09:12:06.580 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T09:12:06.581 INFO:teuthology.task.internal:Checking packages... 2024-09-17T09:12:06.602 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9' 2024-09-17T09:12:06.602 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T09:12:06.602 INFO:teuthology.packaging:ref: None 2024-09-17T09:12:06.602 INFO:teuthology.packaging:tag: None 2024-09-17T09:12:06.603 INFO:teuthology.packaging:branch: reef 2024-09-17T09:12:06.603 INFO:teuthology.packaging:sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T09:12:06.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-17T09:12:06.897 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-855.g19e2a2be 2024-09-17T09:12:06.898 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T09:12:06.905 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T09:12:06.905 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T09:12:06.922 INFO:teuthology.task.internal:Saving configuration 2024-09-17T09:12:06.933 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T09:12:06.939 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T09:12:06.964 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908296', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 09:06:29.398695', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNg6foFUA+2astYu4+zG5YD6PJE4hH6/vTrj4BPo/RR9gcy9EECPhiWltNDj7SlMgvdvfb1gwn+M5JXnaSctvdw='} 2024-09-17T09:12:06.985 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908296', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 09:06:29.400159', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIC59dp1IV3jtFCOoVJ4dWv/eoUa5cTy65J2KbidiVIQciti9qhtHulphxkKC8ko3x8kSBtphfnBTafuq9HkH2o='} 2024-09-17T09:12:06.985 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T09:12:06.991 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-17T09:12:06.992 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-17T09:12:06.992 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T09:12:07.055 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbf85073ac0>, signals=[15]) 2024-09-17T09:12:07.055 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T09:12:07.060 INFO:teuthology.task.internal:Opening connections... 2024-09-17T09:12:07.060 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-09-17T09:12:07.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:12:07.140 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-09-17T09:12:07.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:12:07.218 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T09:12:07.226 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-09-17T09:12:07.244 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-09-17T09:12:07.244 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-09-17T09:12:07.302 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-09-17T09:12:07.302 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-09-17T09:12:07.302 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-09-17T09:12:07.302 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-09-17T09:12:07.302 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-09-17T09:12:07.302 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-09-17T09:12:07.302 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T09:12:07.302 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-09-17T09:12:07.302 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-09-17T09:12:07.303 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T09:12:07.303 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-09-17T09:12:07.303 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T09:12:07.303 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T09:12:07.303 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T09:12:07.304 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-09-17T09:12:07.323 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-09-17T09:12:07.342 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-09-17T09:12:07.342 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-09-17T09:12:07.400 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-09-17T09:12:07.400 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-09-17T09:12:07.400 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-09-17T09:12:07.400 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-09-17T09:12:07.401 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-09-17T09:12:07.401 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-09-17T09:12:07.401 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T09:12:07.401 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-09-17T09:12:07.401 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-09-17T09:12:07.401 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T09:12:07.401 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-09-17T09:12:07.401 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T09:12:07.401 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T09:12:07.401 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T09:12:07.401 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-09-17T09:12:07.423 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T09:12:07.432 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T09:12:07.438 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T09:12:07.438 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-09-17T09:12:07.441 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-09-17T09:12:07.457 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T09:12:07.463 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T09:12:07.463 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-09-17T09:12:07.498 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-09-17T09:12:07.614 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T09:12:07.671 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T09:12:07.684 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T09:12:07.684 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T09:12:07.685 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T09:12:07.685 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T09:12:07.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:12:07.702 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-09-17T09:12:07.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:12:07.703 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-09-17T09:12:07.759 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-503.el9.x86_64 2024-09-17T09:12:07.759 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.14.0-503.el9.x86_64 2024-09-17T09:12:07.759 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2024-09-17T09:12:07.761 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-503.el9.x86_64 2024-09-17T09:12:07.761 INFO:teuthology.task.kernel:Running kernel on smithi157: 5.14.0-503.el9.x86_64 2024-09-17T09:12:07.761 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum install -y kernel 2024-09-17T09:12:10.036 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.3 MB 00:00 2024-09-17T09:12:11.121 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.3 MB 00:01 2024-09-17T09:12:11.671 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-17T09:12:17.213 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-17T09:12:18.622 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 114 kB/s | 19 kB 00:00 2024-09-17T09:12:19.568 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-17T09:12:24.966 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-17T09:12:26.864 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T09:12:26.864 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T09:12:26.959 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-17T09:12:26.968 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-17T09:12:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-17T09:12:26.969 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-17T09:12:26.969 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-17T09:12:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T09:12:26.969 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-17T09:12:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T09:12:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T09:12:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T09:12:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:12:26.969 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-17T09:12:26.969 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-17T09:12:26.970 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-09-17T09:12:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:12:26.970 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 79 M 2024-09-17T09:12:26.970 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 126 M 2024-09-17T09:12:26.970 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-17T09:12:27.108 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 384 kB/s | 36 kB 00:00 2024-09-17T09:12:27.500 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-17T09:12:27.942 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-17T09:12:28.209 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 26 MB/s | 29 MB 00:01 2024-09-17T09:12:28.209 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:12:28.210 INFO:teuthology.orchestra.run.smithi148.stdout:Total 64 MB/s | 79 MB 00:01 2024-09-17T09:12:28.891 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-17T09:12:29.181 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:16 2024-09-17T09:12:29.253 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-17T09:12:29.253 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-17T09:12:30.201 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-17T09:12:30.201 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-17T09:12:32.627 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-17T09:12:33.253 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T09:12:33.343 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:12:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:12:36.252 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:12:36.753 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.5 MB 00:03 2024-09-17T09:12:38.353 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 19 kB 00:00 2024-09-17T09:12:39.249 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-17T09:12:41.088 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:12:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:12:44.722 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-17T09:12:46.062 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:12:46.628 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T09:12:46.629 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T09:12:46.710 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-17T09:12:46.721 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-17T09:12:46.721 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-09-17T09:12:46.721 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-17T09:12:46.721 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-17T09:12:46.722 INFO:teuthology.orchestra.run.smithi157.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T09:12:46.722 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-17T09:12:46.722 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T09:12:46.722 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T09:12:46.722 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T09:12:46.722 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:12:46.722 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-17T09:12:46.722 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-17T09:12:46.722 INFO:teuthology.orchestra.run.smithi157.stdout:Install 4 Packages 2024-09-17T09:12:46.722 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:12:46.723 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 79 M 2024-09-17T09:12:46.723 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 126 M 2024-09-17T09:12:46.723 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-17T09:12:46.856 INFO:teuthology.orchestra.run.smithi157.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 429 kB/s | 36 kB 00:00 2024-09-17T09:12:47.249 INFO:teuthology.orchestra.run.smithi157.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-17T09:12:47.690 INFO:teuthology.orchestra.run.smithi157.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-17T09:12:48.024 INFO:teuthology.orchestra.run.smithi157.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-17T09:12:48.024 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:12:48.024 INFO:teuthology.orchestra.run.smithi157.stdout:Total 61 MB/s | 79 MB 00:01 2024-09-17T09:12:48.704 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-17T09:12:49.051 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-17T09:12:49.051 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-17T09:12:49.995 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-17T09:12:49.996 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-17T09:12:52.421 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-17T09:12:53.126 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T09:12:53.241 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:12:55.991 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:12:56.121 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:13:00.974 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:13:01.012 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:05.842 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:24.905 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:24.906 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T09:13:24.906 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:13:24.906 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:13:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:13:25.766 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-17T09:13:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T09:13:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T09:13:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T09:13:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:13:25.766 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-17T09:13:26.108 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T09:13:26.197 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:26.197 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T09:13:26.197 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T09:13:26.197 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T09:13:26.198 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T09:13:26.198 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T09:13:26.198 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T09:13:26.198 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T09:13:26.257 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:26.257 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T09:13:26.257 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2024-09-17T09:13:26.290 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2024-09-17T09:13:26.290 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2024-09-17T09:13:26.290 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.06 2024-09-17T09:13:26.290 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 82.el9 2024-09-17T09:13:26.290 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2024-09-17T09:13:26.290 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T09:13:26.290 INFO:teuthology.orchestra.run.smithi148.stdout:Group : Unspecified 2024-09-17T09:13:26.290 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 8274670 2024-09-17T09:13:26.290 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2024-09-17T09:13:26.290 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T09:13:26.290 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : builder@centos.org 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : CentOS 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:13:26.291 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T09:13:26.292 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T09:13:26.293 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 5.14.0-508.el9.x86_64 2024-09-17T09:13:26.293 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T09:13:26.933 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-09-17T09:13:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T09:13:28.301 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-09-17T09:13:28.303 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-09-17T09:13:28.318 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.hGzx65HT9V 2024-09-17T09:13:28.319 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hGzx65HT9V 2024-09-17T09:13:28.386 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.hGzx65HT9V 2024-09-17T09:13:28.528 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.hGzx65HT9V is 6KB 2024-09-17T09:13:28.578 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.hGzx65HT9V 2024-09-17T09:13:28.593 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T09:13:28.655 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T09:13:28.655 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T09:13:28.655 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T09:13:28.655 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T09:13:28.656 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T09:13:28.804 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-09-17T09:13:43.947 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:43.987 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:44.278 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:44.278 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T09:13:44.278 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T09:13:44.278 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T09:13:45.145 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T09:13:45.145 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:13:45.145 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-17T09:13:45.145 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:45.145 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T09:13:45.145 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T09:13:45.145 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T09:13:45.145 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:13:45.145 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-17T09:13:45.528 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T09:13:45.621 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:45.622 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T09:13:45.622 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T09:13:45.622 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T09:13:45.622 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T09:13:45.622 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T09:13:45.622 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T09:13:45.622 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T09:13:45.680 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T09:13:45.681 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T09:13:45.681 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2024-09-17T09:13:45.720 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2024-09-17T09:13:45.720 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2024-09-17T09:13:45.720 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.06 2024-09-17T09:13:45.720 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 82.el9 2024-09-17T09:13:45.720 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2024-09-17T09:13:45.720 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T09:13:45.720 INFO:teuthology.orchestra.run.smithi157.stdout:Group : Unspecified 2024-09-17T09:13:45.720 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 8274670 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : builder@centos.org 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : CentOS 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T09:13:45.721 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T09:13:45.722 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T09:13:45.722 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2024-09-17T09:13:45.722 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:13:45.722 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T09:13:45.722 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T09:13:45.723 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 5.14.0-508.el9.x86_64 2024-09-17T09:13:45.723 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T09:13:46.371 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-09-17T09:13:47.652 INFO:teuthology.orchestra.run.smithi157.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T09:13:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-09-17T09:13:47.686 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-09-17T09:13:47.701 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.0LlBAZsLix 2024-09-17T09:13:47.702 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0LlBAZsLix 2024-09-17T09:13:47.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.0LlBAZsLix 2024-09-17T09:13:47.913 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.0LlBAZsLix is 6KB 2024-09-17T09:13:47.964 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.0LlBAZsLix 2024-09-17T09:13:47.980 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T09:13:48.042 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T09:13:48.043 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T09:13:48.043 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T09:13:48.043 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T09:13:48.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T09:13:48.195 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2024-09-17T09:13:58.816 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T09:13:58.817 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-17T09:13:58.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:14:17.277 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-17T09:14:18.199 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T09:14:18.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-17T09:14:18.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:14:26.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-17T09:14:26.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:14:29.345 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-17T09:14:41.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-17T09:14:41.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:14:44.417 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-17T09:14:52.865 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-17T09:14:59.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-17T09:14:59.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:15:01.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-17T09:15:01.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:15:02.493 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-17T09:15:04.957 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-17T09:15:16.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-17T09:15:16.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:15:20.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-17T09:15:20.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:15:20.883 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-17T09:15:21.239 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-17T09:15:21.239 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T09:15:21.239 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-09-17T09:15:21.255 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-508.el9.x86_64 2024-09-17T09:15:21.255 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T09:15:21.255 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T09:15:21.255 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T09:15:22.256 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T09:15:22.256 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T09:15:22.317 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2024-09-17T09:15:22.341 DEBUG:teuthology.parallel:result is None 2024-09-17T09:15:35.361 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-17T09:15:50.363 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-17T09:15:50.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:15:50.670 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-09-17T09:15:51.009 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-17T09:15:51.010 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T09:15:51.010 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-09-17T09:15:51.024 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-508.el9.x86_64 2024-09-17T09:15:51.024 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T09:15:51.025 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T09:15:51.025 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T09:15:52.026 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T09:15:52.026 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T09:15:52.124 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2024-09-17T09:15:52.149 DEBUG:teuthology.parallel:result is None 2024-09-17T09:15:52.149 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T09:15:52.159 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T09:15:52.159 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T09:15:52.162 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T09:15:52.179 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T09:15:52.187 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T09:15:52.193 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T09:15:52.193 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T09:15:52.222 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T09:15:52.519 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T09:15:52.525 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T09:15:52.526 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T09:15:52.528 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T09:15:52.558 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T09:15:52.561 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T09:15:52.569 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T09:15:52.574 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T09:15:52.575 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T09:15:52.582 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T09:15:52.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T09:15:52.618 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T09:15:52.643 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T09:15:52.653 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T09:15:52.653 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T09:15:52.686 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T09:15:52.701 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T09:15:52.786 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T09:15:52.872 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T09:15:52.914 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T09:15:52.958 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:15:52.958 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T09:15:53.024 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T09:15:53.066 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T09:15:53.120 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T09:15:53.153 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T09:15:53.200 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:15:53.200 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T09:15:53.260 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-09-17T09:15:53.263 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-09-17T09:15:53.303 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T09:15:53.355 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T09:15:53.763 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T09:15:53.772 INFO:teuthology.task.internal:Starting timer... 2024-09-17T09:15:53.772 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T09:15:53.782 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T09:15:53.792 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-17T09:15:53.792 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-09-17T09:15:53.836 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-09-17T09:15:53.838 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-09-17T09:15:53.922 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-09-17T09:15:53.924 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T09:15:53.924 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-09-17T09:15:53.945 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-09-17T09:15:53.945 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-09-17T09:15:53.978 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-09-17T09:15:53.978 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi148.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-09-17T09:15:53.978 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T09:15:54.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:15:54.013 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T09:15:54.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:15:54.044 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T09:15:54.044 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-09-17T09:15:54.092 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-09-17T09:15:54.123 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T09:15:54.133 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-17T09:15:54.136 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-17T09:15:54.276 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T09:15:54.288 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T09:15:54.289 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi148.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T09:19:42.518 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi148.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-09-17T09:19:42.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-17T09:19:42.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:19:42.601 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-17T09:19:42.680 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-17T09:19:42.681 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-17T09:19:42.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T09:19:42.758 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-09-17T09:19:42.833 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-17T09:19:42.834 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T09:19:42.845 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T09:19:42.845 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T09:19:42.846 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T09:19:42.848 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T09:19:42.848 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T09:19:42.875 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T09:19:42.890 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T09:19:42.903 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T09:19:42.914 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-09-17T09:19:42.918 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T09:19:42.925 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-09-17T09:19:42.941 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T09:19:42.942 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-09-17T09:19:42.953 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-09-17T09:19:42.954 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T09:19:42.967 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T09:19:42.981 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T09:19:43.012 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-17T09:19:43.016 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T09:19:43.016 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-17T09:19:43.016 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:43.016 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:43.017 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:43.017 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:43.041 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-09-17T09:19:43.044 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T09:19:43.045 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-09-17T09:19:43.045 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:43.045 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:43.045 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:43.045 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:19:43.046 INFO:teuthology.run_tasks:Running task pexec... 2024-09-17T09:19:43.055 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-17T09:19:43.056 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi148.front.sepia.ceph.com 2024-09-17T09:19:43.056 DEBUG:teuthology.orchestra.run.smithi148:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T09:19:43.056 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi157.front.sepia.ceph.com 2024-09-17T09:19:43.057 DEBUG:teuthology.orchestra.run.smithi157:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-17T09:19:43.411 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-17T09:19:43.425 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout:Removing: 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout:Removing unused dependencies: 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout:Remove 2 Packages 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout:Freed space: 5.5 M 2024-09-17T09:19:43.426 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-17T09:19:43.430 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-17T09:19:43.430 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-17T09:19:43.439 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-17T09:19:43.451 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-17T09:19:43.451 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-09-17T09:19:43.451 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-17T09:19:43.451 INFO:teuthology.orchestra.run.smithi157.stdout:Removing: 2024-09-17T09:19:43.451 INFO:teuthology.orchestra.run.smithi157.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-17T09:19:43.451 INFO:teuthology.orchestra.run.smithi157.stdout:Removing unused dependencies: 2024-09-17T09:19:43.451 INFO:teuthology.orchestra.run.smithi157.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-17T09:19:43.451 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:19:43.452 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-17T09:19:43.452 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-17T09:19:43.452 INFO:teuthology.orchestra.run.smithi157.stdout:Remove 2 Packages 2024-09-17T09:19:43.452 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:19:43.452 INFO:teuthology.orchestra.run.smithi157.stdout:Freed space: 5.5 M 2024-09-17T09:19:43.452 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-17T09:19:43.455 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-17T09:19:43.456 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-17T09:19:43.480 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-17T09:19:43.480 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-17T09:19:43.517 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-17T09:19:43.517 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-17T09:19:43.620 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-17T09:19:43.652 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-17T09:19:43.693 INFO:teuthology.orchestra.run.smithi148.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T09:19:43.740 INFO:teuthology.orchestra.run.smithi157.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-17T09:19:43.752 INFO:teuthology.orchestra.run.smithi148.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T09:19:43.796 INFO:teuthology.orchestra.run.smithi157.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T09:19:44.278 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T09:19:44.279 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T09:19:44.333 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-17T09:19:44.333 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-17T09:19:44.579 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T09:19:44.580 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:19:44.580 INFO:teuthology.orchestra.run.smithi148.stdout:Removed: 2024-09-17T09:19:44.580 INFO:teuthology.orchestra.run.smithi148.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T09:19:44.580 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:19:44.580 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-17T09:19:44.606 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-17T09:19:44.606 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:19:44.606 INFO:teuthology.orchestra.run.smithi157.stdout:Removed: 2024-09-17T09:19:44.606 INFO:teuthology.orchestra.run.smithi157.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T09:19:44.606 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:19:44.606 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-17T09:19:45.453 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:51 ago on Tue 17 Sep 2024 09:18:54 AM UTC. 2024-09-17T09:19:45.458 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:51 ago on Tue 17 Sep 2024 09:18:54 AM UTC. 2024-09-17T09:19:45.589 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T09:19:45.590 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:19:45.591 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-17T09:19:45.591 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-17T09:19:45.591 INFO:teuthology.orchestra.run.smithi148.stdout:Install 6 Packages 2024-09-17T09:19:45.591 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:19:45.593 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-09-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-17T09:19:45.594 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi157.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi157.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi157.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi157.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-17T09:19:45.595 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-17T09:19:45.596 INFO:teuthology.orchestra.run.smithi157.stdout:Install 6 Packages 2024-09-17T09:19:45.596 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:19:45.599 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 2.0 M 2024-09-17T09:19:45.599 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 9.2 M 2024-09-17T09:19:45.599 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-17T09:19:45.599 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 2.0 M 2024-09-17T09:19:45.599 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 9.2 M 2024-09-17T09:19:45.600 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-17T09:19:45.785 INFO:teuthology.orchestra.run.smithi157.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 528 kB/s | 44 kB 00:00 2024-09-17T09:19:45.803 INFO:teuthology.orchestra.run.smithi148.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 428 kB/s | 44 kB 00:00 2024-09-17T09:19:45.811 INFO:teuthology.orchestra.run.smithi157.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 968 kB/s | 106 kB 00:00 2024-09-17T09:19:45.844 INFO:teuthology.orchestra.run.smithi157.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 1.2 MB/s | 72 kB 00:00 2024-09-17T09:19:45.861 INFO:teuthology.orchestra.run.smithi148.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 653 kB/s | 106 kB 00:00 2024-09-17T09:19:45.869 INFO:teuthology.orchestra.run.smithi157.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.4 MB/s | 84 kB 00:00 2024-09-17T09:19:45.887 INFO:teuthology.orchestra.run.smithi148.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 858 kB/s | 72 kB 00:00 2024-09-17T09:19:45.903 INFO:teuthology.orchestra.run.smithi157.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.4 MB/s | 906 kB 00:00 2024-09-17T09:19:45.912 INFO:teuthology.orchestra.run.smithi148.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-17T09:19:45.945 INFO:teuthology.orchestra.run.smithi157.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 8.1 MB/s | 837 kB 00:00 2024-09-17T09:19:45.945 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:19:45.946 INFO:teuthology.orchestra.run.smithi157.stdout:Total 5.8 MB/s | 2.0 MB 00:00 2024-09-17T09:19:46.020 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-17T09:19:46.029 INFO:teuthology.orchestra.run.smithi148.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.7 MB/s | 906 kB 00:00 2024-09-17T09:19:46.034 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-17T09:19:46.034 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-17T09:19:46.088 INFO:teuthology.orchestra.run.smithi148.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.1 MB/s | 837 kB 00:00 2024-09-17T09:19:46.088 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:19:46.088 INFO:teuthology.orchestra.run.smithi148.stdout:Total 4.1 MB/s | 2.0 MB 00:00 2024-09-17T09:19:46.161 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-17T09:19:46.175 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-17T09:19:46.175 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-17T09:19:46.318 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-17T09:19:46.318 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-17T09:19:46.458 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-17T09:19:46.459 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-17T09:19:46.734 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-17T09:19:46.823 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-17T09:19:46.857 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-17T09:19:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-17T09:19:46.885 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-17T09:19:46.965 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-17T09:19:46.999 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-17T09:19:47.028 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-17T09:19:47.092 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-17T09:19:47.125 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-17T09:19:47.236 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-17T09:19:47.276 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-17T09:19:47.566 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-17T09:19:47.594 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-17T09:19:47.720 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-17T09:19:47.745 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-17T09:19:48.567 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-17T09:19:48.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-17T09:19:48.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-17T09:19:48.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-17T09:19:48.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-17T09:19:48.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-17T09:19:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-17T09:19:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-17T09:19:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-17T09:19:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-17T09:19:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-17T09:19:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-17T09:19:48.879 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-17T09:19:48.879 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:19:48.880 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-17T09:19:48.880 INFO:teuthology.orchestra.run.smithi157.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T09:19:48.880 INFO:teuthology.orchestra.run.smithi157.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T09:19:48.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T09:19:48.880 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:19:48.880 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-17T09:19:49.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-17T09:19:49.061 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:19:49.061 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-17T09:19:49.061 INFO:teuthology.orchestra.run.smithi148.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-17T09:19:49.061 INFO:teuthology.orchestra.run.smithi148.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-17T09:19:49.061 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-17T09:19:49.061 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:19:49.061 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-17T09:19:49.175 DEBUG:teuthology.parallel:result is None 2024-09-17T09:19:49.422 DEBUG:teuthology.parallel:result is None 2024-09-17T09:19:49.422 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-09-17T09:19:49.434 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-09-17T09:19:49.434 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:19:49.434 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:19:49.451 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T09:19:49.451 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-09-17T09:19:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T09:19:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:19:49.507 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 983 Links: 1 2024-09-17T09:19:49.507 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:19:49.507 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:19:49.507 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-17 09:19:48.066702222 +0000 2024-09-17T09:19:49.507 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-17 09:19:47.828705528 +0000 2024-09-17T09:19:49.508 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-17 09:19:47.828705528 +0000 2024-09-17T09:19:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-17 09:19:47.828705528 +0000 2024-09-17T09:19:49.508 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T09:19:49.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-17T09:19:49.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-17T09:19:49.572 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000195921 s, 2.6 MB/s 2024-09-17T09:19:49.573 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T09:19:49.631 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-09-17T09:19:49.686 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T09:19:49.686 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:19:49.686 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-17T09:19:49.686 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:19:49.687 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:19:49.687 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-17 09:19:48.066702222 +0000 2024-09-17T09:19:49.687 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-17 09:19:47.825705600 +0000 2024-09-17T09:19:49.687 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-17 09:19:47.825705600 +0000 2024-09-17T09:19:49.687 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-17 09:19:47.825705600 +0000 2024-09-17T09:19:49.687 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T09:19:49.750 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-17T09:19:49.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-17T09:19:49.751 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000191889 s, 2.7 MB/s 2024-09-17T09:19:49.752 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T09:19:49.809 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-09-17T09:19:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T09:19:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:19:49.864 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-17T09:19:49.864 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:19:49.864 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:19:49.865 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-17 09:19:48.066702222 +0000 2024-09-17T09:19:49.865 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-17 09:19:47.825705600 +0000 2024-09-17T09:19:49.865 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-17 09:19:47.825705600 +0000 2024-09-17T09:19:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-17 09:19:47.825705600 +0000 2024-09-17T09:19:49.865 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T09:19:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-17T09:19:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-17T09:19:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000208615 s, 2.5 MB/s 2024-09-17T09:19:49.933 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T09:19:49.989 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-09-17T09:19:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T09:19:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:19:50.046 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-17T09:19:50.046 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:19:50.046 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:19:50.046 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-17 09:19:48.067702212 +0000 2024-09-17T09:19:50.046 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-17 09:19:47.826705576 +0000 2024-09-17T09:19:50.046 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-17 09:19:47.826705576 +0000 2024-09-17T09:19:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-17 09:19:47.826705576 +0000 2024-09-17T09:19:50.046 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T09:19:50.109 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-17T09:19:50.109 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-17T09:19:50.109 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000205045 s, 2.5 MB/s 2024-09-17T09:19:50.110 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T09:19:50.167 DEBUG:teuthology.orchestra.run.smithi148:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-17T09:19:50.312 INFO:teuthology.orchestra.run.smithi148.stdout:loop 2024-09-17T09:19:50.313 INFO:tasks.nvme_loop:Connecting nvme_loop smithi148:/dev/vg_nvme/lv_1... 2024-09-17T09:19:50.313 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-17T09:19:50.346 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-09-17T09:19:50.378 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/vg_nvme/lv_11 2024-09-17T09:19:50.418 INFO:teuthology.orchestra.run.smithi148.stdout:connecting to device: nvme1 2024-09-17T09:19:50.420 INFO:tasks.nvme_loop:Connecting nvme_loop smithi148:/dev/vg_nvme/lv_2... 2024-09-17T09:19:50.420 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-17T09:19:50.495 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-09-17T09:19:50.526 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/vg_nvme/lv_21 2024-09-17T09:19:50.558 INFO:teuthology.orchestra.run.smithi148.stdout:connecting to device: nvme2 2024-09-17T09:19:50.559 INFO:tasks.nvme_loop:Connecting nvme_loop smithi148:/dev/vg_nvme/lv_3... 2024-09-17T09:19:50.559 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-17T09:19:50.635 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-09-17T09:19:50.666 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/vg_nvme/lv_31 2024-09-17T09:19:50.701 INFO:teuthology.orchestra.run.smithi148.stdout:connecting to device: nvme3 2024-09-17T09:19:50.703 INFO:tasks.nvme_loop:Connecting nvme_loop smithi148:/dev/vg_nvme/lv_4... 2024-09-17T09:19:50.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-17T09:19:50.777 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-09-17T09:19:50.812 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/vg_nvme/lv_41 2024-09-17T09:19:50.854 INFO:teuthology.orchestra.run.smithi148.stdout:connecting to device: nvme4 2024-09-17T09:19:50.854 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:19:50.854 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:19:50.909 DEBUG:teuthology.orchestra.run.smithi148:> sudo nvme list -o json 2024-09-17T09:19:50.976 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-17T09:19:50.976 INFO:teuthology.orchestra.run.smithi148.stdout: "Devices":[ 2024-09-17T09:19:50.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-17T09:19:50.976 INFO:teuthology.orchestra.run.smithi148.stdout: "NameSpace":1, 2024-09-17T09:19:50.976 INFO:teuthology.orchestra.run.smithi148.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: "GenericPath":"/dev/ng4n1", 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: "ModelNumber":"Linux", 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: "SerialNumber":"0a446a8efc652f5d297c", 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: "UsedBytes":95995035648, 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: "MaximumLBA":187490304, 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: "PhysicalSize":95995035648, 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: "SectorSize":512 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: "NameSpace":1, 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-17T09:19:50.977 INFO:teuthology.orchestra.run.smithi148.stdout: "GenericPath":"/dev/ng3n1", 2024-09-17T09:19:50.978 INFO:teuthology.orchestra.run.smithi148.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:19:50.978 INFO:teuthology.orchestra.run.smithi148.stdout: "ModelNumber":"Linux", 2024-09-17T09:19:50.978 INFO:teuthology.orchestra.run.smithi148.stdout: "SerialNumber":"12b1430a50ac941ee93e", 2024-09-17T09:19:50.978 INFO:teuthology.orchestra.run.smithi148.stdout: "UsedBytes":95995035648, 2024-09-17T09:19:50.978 INFO:teuthology.orchestra.run.smithi148.stdout: "MaximumLBA":187490304, 2024-09-17T09:19:50.978 INFO:teuthology.orchestra.run.smithi148.stdout: "PhysicalSize":95995035648, 2024-09-17T09:19:50.978 INFO:teuthology.orchestra.run.smithi148.stdout: "SectorSize":512 2024-09-17T09:19:50.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: "NameSpace":1, 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: "GenericPath":"/dev/ng2n1", 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: "ModelNumber":"Linux", 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: "SerialNumber":"4379e416df77cf91daa0", 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: "UsedBytes":95995035648, 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: "MaximumLBA":187490304, 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: "PhysicalSize":95995035648, 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: "SectorSize":512 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-17T09:19:50.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-17T09:19:50.980 INFO:teuthology.orchestra.run.smithi148.stdout: "NameSpace":1, 2024-09-17T09:19:50.980 INFO:teuthology.orchestra.run.smithi148.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-17T09:19:50.980 INFO:teuthology.orchestra.run.smithi148.stdout: "GenericPath":"/dev/ng1n1", 2024-09-17T09:19:50.980 INFO:teuthology.orchestra.run.smithi148.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:19:50.980 INFO:teuthology.orchestra.run.smithi148.stdout: "ModelNumber":"Linux", 2024-09-17T09:19:50.980 INFO:teuthology.orchestra.run.smithi148.stdout: "SerialNumber":"da21824e67705baa6884", 2024-09-17T09:19:50.980 INFO:teuthology.orchestra.run.smithi148.stdout: "UsedBytes":95995035648, 2024-09-17T09:19:50.980 INFO:teuthology.orchestra.run.smithi148.stdout: "MaximumLBA":187490304, 2024-09-17T09:19:50.980 INFO:teuthology.orchestra.run.smithi148.stdout: "PhysicalSize":95995035648, 2024-09-17T09:19:50.980 INFO:teuthology.orchestra.run.smithi148.stdout: "SectorSize":512 2024-09-17T09:19:50.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-17T09:19:50.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-17T09:19:50.981 INFO:teuthology.orchestra.run.smithi148.stdout: "NameSpace":1, 2024-09-17T09:19:50.981 INFO:teuthology.orchestra.run.smithi148.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-17T09:19:50.981 INFO:teuthology.orchestra.run.smithi148.stdout: "GenericPath":"/dev/ng0n1", 2024-09-17T09:19:50.981 INFO:teuthology.orchestra.run.smithi148.stdout: "Firmware":"8DV101H0", 2024-09-17T09:19:50.981 INFO:teuthology.orchestra.run.smithi148.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-17T09:19:50.981 INFO:teuthology.orchestra.run.smithi148.stdout: "SerialNumber":"CVFT6233005G400BGN", 2024-09-17T09:19:50.981 INFO:teuthology.orchestra.run.smithi148.stdout: "UsedBytes":400088457216, 2024-09-17T09:19:50.982 INFO:teuthology.orchestra.run.smithi148.stdout: "MaximumLBA":781422768, 2024-09-17T09:19:50.982 INFO:teuthology.orchestra.run.smithi148.stdout: "PhysicalSize":400088457216, 2024-09-17T09:19:50.982 INFO:teuthology.orchestra.run.smithi148.stdout: "SectorSize":512 2024-09-17T09:19:50.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-17T09:19:50.982 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-17T09:19:50.982 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-17T09:19:50.983 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T09:19:50.983 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:19:50.983 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/scratch_devs 2024-09-17T09:19:51.048 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:19:51.048 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:19:51.069 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T09:19:51.069 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-09-17T09:19:51.128 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T09:19:51.128 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:19:51.128 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 952 Links: 1 2024-09-17T09:19:51.128 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:19:51.128 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:19:51.129 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-17 09:19:47.919999802 +0000 2024-09-17T09:19:51.129 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-17 09:19:47.674000156 +0000 2024-09-17T09:19:51.129 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-17 09:19:47.674000156 +0000 2024-09-17T09:19:51.129 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-17 09:19:47.674000156 +0000 2024-09-17T09:19:51.129 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T09:19:51.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-17T09:19:51.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-17T09:19:51.198 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000116202 s, 4.4 MB/s 2024-09-17T09:19:51.199 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T09:19:51.258 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-09-17T09:19:51.316 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T09:19:51.316 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:19:51.316 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 953 Links: 1 2024-09-17T09:19:51.316 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:19:51.316 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:19:51.316 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-17 09:19:47.919999802 +0000 2024-09-17T09:19:51.316 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-17 09:19:47.674000156 +0000 2024-09-17T09:19:51.316 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-17 09:19:47.674000156 +0000 2024-09-17T09:19:51.316 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-17 09:19:47.674000156 +0000 2024-09-17T09:19:51.317 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T09:19:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-17T09:19:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-17T09:19:51.382 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000136275 s, 3.8 MB/s 2024-09-17T09:19:51.383 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T09:19:51.442 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-09-17T09:19:51.498 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T09:19:51.499 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:19:51.499 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 951 Links: 1 2024-09-17T09:19:51.499 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:19:51.499 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:19:51.499 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-17 09:19:47.920999972 +0000 2024-09-17T09:19:51.499 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-17 09:19:47.674000156 +0000 2024-09-17T09:19:51.499 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-17 09:19:47.674000156 +0000 2024-09-17T09:19:51.499 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-17 09:19:47.674000156 +0000 2024-09-17T09:19:51.499 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T09:19:51.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-17T09:19:51.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-17T09:19:51.564 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000103634 s, 4.9 MB/s 2024-09-17T09:19:51.565 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T09:19:51.621 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-09-17T09:19:51.678 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T09:19:51.678 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T09:19:51.678 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-09-17T09:19:51.678 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T09:19:51.678 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T09:19:51.678 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-17 09:19:47.920999972 +0000 2024-09-17T09:19:51.678 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-17 09:19:47.675000138 +0000 2024-09-17T09:19:51.678 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-17 09:19:47.675000138 +0000 2024-09-17T09:19:51.678 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-17 09:19:47.675000138 +0000 2024-09-17T09:19:51.679 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T09:19:51.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-17T09:19:51.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-17T09:19:51.742 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000110742 s, 4.6 MB/s 2024-09-17T09:19:51.743 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T09:19:51.800 DEBUG:teuthology.orchestra.run.smithi157:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-17T09:19:51.925 INFO:teuthology.orchestra.run.smithi157.stdout:loop 2024-09-17T09:19:51.926 INFO:tasks.nvme_loop:Connecting nvme_loop smithi157:/dev/vg_nvme/lv_1... 2024-09-17T09:19:51.926 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-17T09:19:51.959 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-17T09:19:51.989 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/vg_nvme/lv_11 2024-09-17T09:19:52.035 INFO:teuthology.orchestra.run.smithi157.stdout:connecting to device: nvme1 2024-09-17T09:19:52.037 INFO:tasks.nvme_loop:Connecting nvme_loop smithi157:/dev/vg_nvme/lv_2... 2024-09-17T09:19:52.038 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-17T09:19:52.112 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-17T09:19:52.148 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/vg_nvme/lv_21 2024-09-17T09:19:52.179 INFO:teuthology.orchestra.run.smithi157.stdout:connecting to device: nvme2 2024-09-17T09:19:52.181 INFO:tasks.nvme_loop:Connecting nvme_loop smithi157:/dev/vg_nvme/lv_3... 2024-09-17T09:19:52.182 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-17T09:19:52.256 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-17T09:19:52.286 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/vg_nvme/lv_31 2024-09-17T09:19:52.317 INFO:teuthology.orchestra.run.smithi157.stdout:connecting to device: nvme3 2024-09-17T09:19:52.323 INFO:tasks.nvme_loop:Connecting nvme_loop smithi157:/dev/vg_nvme/lv_4... 2024-09-17T09:19:52.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-17T09:19:52.397 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-17T09:19:52.428 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/vg_nvme/lv_41 2024-09-17T09:19:52.462 INFO:teuthology.orchestra.run.smithi157.stdout:connecting to device: nvme4 2024-09-17T09:19:52.465 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:19:52.465 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:19:52.521 DEBUG:teuthology.orchestra.run.smithi157:> sudo nvme list -o json 2024-09-17T09:19:52.590 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-17T09:19:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "Devices":[ 2024-09-17T09:19:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: { 2024-09-17T09:19:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "NameSpace":1, 2024-09-17T09:19:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-17T09:19:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "GenericPath":"/dev/ng4n1", 2024-09-17T09:19:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:19:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "ModelNumber":"Linux", 2024-09-17T09:19:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "SerialNumber":"fd032744ff9608c07997", 2024-09-17T09:19:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "UsedBytes":95995035648, 2024-09-17T09:19:52.592 INFO:teuthology.orchestra.run.smithi157.stdout: "MaximumLBA":187490304, 2024-09-17T09:19:52.592 INFO:teuthology.orchestra.run.smithi157.stdout: "PhysicalSize":95995035648, 2024-09-17T09:19:52.592 INFO:teuthology.orchestra.run.smithi157.stdout: "SectorSize":512 2024-09-17T09:19:52.592 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-17T09:19:52.592 INFO:teuthology.orchestra.run.smithi157.stdout: { 2024-09-17T09:19:52.592 INFO:teuthology.orchestra.run.smithi157.stdout: "NameSpace":1, 2024-09-17T09:19:52.592 INFO:teuthology.orchestra.run.smithi157.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-17T09:19:52.592 INFO:teuthology.orchestra.run.smithi157.stdout: "GenericPath":"/dev/ng3n1", 2024-09-17T09:19:52.592 INFO:teuthology.orchestra.run.smithi157.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:19:52.592 INFO:teuthology.orchestra.run.smithi157.stdout: "ModelNumber":"Linux", 2024-09-17T09:19:52.592 INFO:teuthology.orchestra.run.smithi157.stdout: "SerialNumber":"2ed25dbc61f73a85ec5b", 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "UsedBytes":95995035648, 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "MaximumLBA":187490304, 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "PhysicalSize":95995035648, 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "SectorSize":512 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: { 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "NameSpace":1, 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "GenericPath":"/dev/ng2n1", 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "ModelNumber":"Linux", 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "SerialNumber":"67094b2ea30257741a9e", 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "UsedBytes":95995035648, 2024-09-17T09:19:52.593 INFO:teuthology.orchestra.run.smithi157.stdout: "MaximumLBA":187490304, 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "PhysicalSize":95995035648, 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "SectorSize":512 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: { 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "NameSpace":1, 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "GenericPath":"/dev/ng1n1", 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "Firmware":"5.14.0-5", 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "ModelNumber":"Linux", 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "SerialNumber":"6bc07644d91991691c5d", 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "UsedBytes":95995035648, 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "MaximumLBA":187490304, 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "PhysicalSize":95995035648, 2024-09-17T09:19:52.594 INFO:teuthology.orchestra.run.smithi157.stdout: "SectorSize":512 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: { 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: "NameSpace":1, 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: "GenericPath":"/dev/ng0n1", 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: "Firmware":"E2010325", 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: "ModelNumber":"INTEL SSDPED1D480GA", 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: "SerialNumber":"PHMB7476002V480DGN", 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: "UsedBytes":480103981056, 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: "MaximumLBA":937703088, 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: "PhysicalSize":480103981056, 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: "SectorSize":512 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-09-17T09:19:52.595 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-17T09:19:52.596 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T09:19:52.597 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:19:52.597 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/scratch_devs 2024-09-17T09:19:52.663 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-17T09:19:52.758 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': True}, 'global': {'mon election default strategy': 3}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'MON_DOWN', 'mons down', 'mon down', 'out of quorum', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'CEPHADM_AGENT_DOWN'], 'log-only-match': ['CEPHADM_'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'} 2024-09-17T09:19:52.758 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T09:19:52.759 INFO:tasks.cephadm:Cluster fsid is ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:19:52.759 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-17T09:19:52.759 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.148', 'mon.c': '[v2:172.21.15.148:3301,v1:172.21.15.148:6790]', 'mon.b': '172.21.15.157'} 2024-09-17T09:19:52.759 INFO:tasks.cephadm:First mon is mon.a on smithi148 2024-09-17T09:19:52.759 INFO:tasks.cephadm:First mgr is y 2024-09-17T09:19:52.759 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-17T09:19:52.759 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2024-09-17T09:19:52.790 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2024-09-17T09:19:52.816 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-17T09:19:52.816 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T09:19:52.960 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56094/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-855-g19e2a2be', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-855.g19e2a2be'}, 'url': 'https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-16 16:29:21.739533', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-17T09:19:53.098 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref reef, sha1 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T09:19:53.099 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm 2024-09-17T09:19:53.100 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-17T09:19:53.411 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 17 09:19 /home/ubuntu/cephtest/cephadm 2024-09-17T09:19:53.411 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-17T09:19:53.753 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 17 09:19 /home/ubuntu/cephtest/cephadm 2024-09-17T09:19:53.753 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-17T09:19:53.772 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-17T09:19:53.806 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 on all hosts... 2024-09-17T09:19:53.806 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T09:19:53.815 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 pull 2024-09-17T09:19:55.345 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T09:19:55.367 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T09:20:27.603 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-09-17T09:20:27.604 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T09:20:27.604 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T09:20:27.604 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2024-09-17T09:20:27.604 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T09:20:27.604 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-09-17T09:20:27.604 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-09-17T09:20:43.021 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-17T09:20:43.021 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable)", 2024-09-17T09:20:43.021 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89", 2024-09-17T09:20:43.021 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2024-09-17T09:20:43.021 INFO:teuthology.orchestra.run.smithi148.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9" 2024-09-17T09:20:43.022 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-17T09:20:43.022 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-17T09:20:43.039 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2024-09-17T09:20:43.091 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2024-09-17T09:20:43.125 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2024-09-17T09:20:43.150 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2024-09-17T09:20:43.195 INFO:tasks.cephadm:Writing seed config... 2024-09-17T09:20:43.197 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-17T09:20:43.197 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-17T09:20:43.197 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = True 2024-09-17T09:20:43.197 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-17T09:20:43.197 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-17T09:20:43.197 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-17T09:20:43.197 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-17T09:20:43.197 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-17T09:20:43.197 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-17T09:20:43.197 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-17T09:20:43.198 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:20:43.198 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-17T09:20:43.213 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ff0bb856-74d5-11ef-bceb-c7b262605968 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = True [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-17T09:20:43.214 DEBUG:teuthology.orchestra.run.smithi148:mon.a> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.a.service 2024-09-17T09:20:43.258 DEBUG:teuthology.orchestra.run.smithi148:mgr.y> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.y.service 2024-09-17T09:20:43.301 INFO:tasks.cephadm:Bootstrapping... 2024-09-17T09:20:43.301 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 -v bootstrap --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.148 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:20:43.453 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-17T09:20:43.453 INFO:teuthology.orchestra.run.smithi148.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', '-v', 'bootstrap', '--fsid', 'ff0bb856-74d5-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.148', '--skip-admin-label'] 2024-09-17T09:20:43.476 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/podman: stdout 5.2.2 2024-09-17T09:20:43.476 INFO:teuthology.orchestra.run.smithi148.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-17T09:20:43.476 INFO:teuthology.orchestra.run.smithi148.stdout:Verifying podman|docker is present... 2024-09-17T09:20:43.498 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/podman: stdout 5.2.2 2024-09-17T09:20:43.498 INFO:teuthology.orchestra.run.smithi148.stdout:Verifying lvm2 is present... 2024-09-17T09:20:43.498 INFO:teuthology.orchestra.run.smithi148.stdout:Verifying time synchronization is in place... 2024-09-17T09:20:43.505 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T09:20:43.505 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T09:20:43.511 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T09:20:43.511 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout inactive 2024-09-17T09:20:43.518 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout enabled 2024-09-17T09:20:43.524 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout active 2024-09-17T09:20:43.524 INFO:teuthology.orchestra.run.smithi148.stdout:Unit chronyd.service is enabled and running 2024-09-17T09:20:43.524 INFO:teuthology.orchestra.run.smithi148.stdout:Repeating the final host check... 2024-09-17T09:20:43.546 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/podman: stdout 5.2.2 2024-09-17T09:20:43.546 INFO:teuthology.orchestra.run.smithi148.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-17T09:20:43.546 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl is present 2024-09-17T09:20:43.546 INFO:teuthology.orchestra.run.smithi148.stdout:lvcreate is present 2024-09-17T09:20:43.553 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-17T09:20:43.553 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-17T09:20:43.559 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-17T09:20:43.559 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout inactive 2024-09-17T09:20:43.565 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout enabled 2024-09-17T09:20:43.572 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout active 2024-09-17T09:20:43.572 INFO:teuthology.orchestra.run.smithi148.stdout:Unit chronyd.service is enabled and running 2024-09-17T09:20:43.572 INFO:teuthology.orchestra.run.smithi148.stdout:Host looks OK 2024-09-17T09:20:43.572 INFO:teuthology.orchestra.run.smithi148.stdout:Cluster fsid: ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:20:43.572 INFO:teuthology.orchestra.run.smithi148.stdout:Acquiring lock 140177394167968 on /run/cephadm/ff0bb856-74d5-11ef-bceb-c7b262605968.lock 2024-09-17T09:20:43.572 INFO:teuthology.orchestra.run.smithi148.stdout:Lock 140177394167968 acquired on /run/cephadm/ff0bb856-74d5-11ef-bceb-c7b262605968.lock 2024-09-17T09:20:43.573 INFO:teuthology.orchestra.run.smithi148.stdout:Verifying IP 172.21.15.148 port 3300 ... 2024-09-17T09:20:43.573 INFO:teuthology.orchestra.run.smithi148.stdout:Verifying IP 172.21.15.148 port 6789 ... 2024-09-17T09:20:43.573 INFO:teuthology.orchestra.run.smithi148.stdout:Base mon IP(s) is [172.21.15.148:3300, 172.21.15.148:6789], mon addrv is [v2:172.21.15.148:3300,v1:172.21.15.148:6789] 2024-09-17T09:20:43.576 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.148 metric 100 2024-09-17T09:20:43.576 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.148 metric 100 2024-09-17T09:20:43.579 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-17T09:20:43.579 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-17T09:20:43.579 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-17T09:20:43.581 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-17T09:20:43.581 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-17T09:20:43.581 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T09:20:43.581 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-17T09:20:43.581 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2024-09-17T09:20:43.581 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-17T09:20:43.582 INFO:teuthology.orchestra.run.smithi148.stdout:Mon IP `172.21.15.148` is in CIDR network `172.21.0.0/20` 2024-09-17T09:20:43.582 INFO:teuthology.orchestra.run.smithi148.stdout:Mon IP `172.21.15.148` is in CIDR network `172.21.0.0/20` 2024-09-17T09:20:43.582 INFO:teuthology.orchestra.run.smithi148.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-17T09:20:43.582 INFO:teuthology.orchestra.run.smithi148.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-17T09:20:43.583 INFO:teuthology.orchestra.run.smithi148.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T09:20:44.040 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/podman: stdout 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T09:20:44.040 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9... 2024-09-17T09:20:44.040 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/podman: stderr Getting image source signatures 2024-09-17T09:20:44.040 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/podman: stderr Copying blob sha256:d8a707c21dc06a64315160cb2d8e524fa91943a344731b2a930ee3a716bebbdd 2024-09-17T09:20:44.041 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-17T09:20:44.041 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/podman: stderr Copying config sha256:980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T09:20:44.041 INFO:teuthology.orchestra.run.smithi148.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-17T09:20:45.244 INFO:teuthology.orchestra.run.smithi148.stdout:ceph: stdout ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable) 2024-09-17T09:20:45.245 INFO:teuthology.orchestra.run.smithi148.stdout:Ceph version: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable) 2024-09-17T09:20:45.245 INFO:teuthology.orchestra.run.smithi148.stdout:Extracting ceph user uid/gid from container image... 2024-09-17T09:20:46.242 INFO:teuthology.orchestra.run.smithi148.stdout:stat: stdout 167 167 2024-09-17T09:20:46.243 INFO:teuthology.orchestra.run.smithi148.stdout:Creating initial keys... 2024-09-17T09:20:47.180 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph-authtool: stdout AQDuSelmSVXaIhAAqy9mFjV18lc9GeAo1clAcg== 2024-09-17T09:20:48.178 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph-authtool: stdout AQDvSelmiakZHhAACwSeobeu2rjnSKUOI86DfA== 2024-09-17T09:20:49.067 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph-authtool: stdout AQDwSelm1NZlHRAA9FQeG4DCmTci8sPxB/XqCw== 2024-09-17T09:20:49.068 INFO:teuthology.orchestra.run.smithi148.stdout:Creating initial monmap... 2024-09-17T09:20:50.006 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T09:20:50.006 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-17T09:20:50.007 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:20:50.007 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T09:20:50.007 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool for a [v2:172.21.15.148:3300,v1:172.21.15.148:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-17T09:20:50.007 INFO:teuthology.orchestra.run.smithi148.stdout:setting min_mon_release = pacific 2024-09-17T09:20:50.007 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/monmaptool: set fsid to ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:20:50.007 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-17T09:20:50.007 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:20:50.007 INFO:teuthology.orchestra.run.smithi148.stdout:Creating mon... 2024-09-17T09:20:50.929 INFO:teuthology.orchestra.run.smithi148.stdout:create mon.a on 2024-09-17T09:20:51.300 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-17T09:20:51.523 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ff0bb856-74d5-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-ff0bb856-74d5-11ef-bceb-c7b262605968.target. 2024-09-17T09:20:51.523 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ff0bb856-74d5-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-ff0bb856-74d5-11ef-bceb-c7b262605968.target. 2024-09-17T09:20:51.756 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.a 2024-09-17T09:20:51.756 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-17T09:20:51.944 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ff0bb856-74d5-11ef-bceb-c7b262605968.target.wants/ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@.service. 2024-09-17T09:20:52.495 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T09:20:52.495 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout disabled 2024-09-17T09:20:52.503 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T09:20:52.503 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout inactive 2024-09-17T09:20:52.504 INFO:teuthology.orchestra.run.smithi148.stdout:firewalld.service is not enabled 2024-09-17T09:20:52.504 INFO:teuthology.orchestra.run.smithi148.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T09:20:52.504 INFO:teuthology.orchestra.run.smithi148.stdout:Waiting for mon to start... 2024-09-17T09:20:52.504 INFO:teuthology.orchestra.run.smithi148.stdout:Waiting for mon... 2024-09-17T09:20:53.570 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:53 smithi148 ceph-mon[25283]: from='client.? 172.21.15.148:0/2110412302' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-17T09:20:53.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout cluster: 2024-09-17T09:20:53.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout id: ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:20:53.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-17T09:20:53.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:20:53.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout services: 2024-09-17T09:20:53.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.55528s) 2024-09-17T09:20:53.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-17T09:20:53.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-17T09:20:53.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:20:53.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout data: 2024-09-17T09:20:53.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-17T09:20:53.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-17T09:20:53.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-17T09:20:53.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout pgs: 2024-09-17T09:20:53.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:20:53.574 INFO:teuthology.orchestra.run.smithi148.stdout:mon is available 2024-09-17T09:20:53.574 INFO:teuthology.orchestra.run.smithi148.stdout:Assimilating anything we can from ceph.conf... 2024-09-17T09:20:54.609 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:54 smithi148 ceph-mon[25283]: from='client.? 172.21.15.148:0/2056667155' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-17T09:20:54.609 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:54 smithi148 ceph-mon[25283]: from='client.? 172.21.15.148:0/2056667155' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-17T09:20:54.612 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:20:54.612 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T09:20:54.612 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout fsid = ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:20:54.612 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.148:3300,v1:172.21.15.148:6789] 2024-09-17T09:20:54.612 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T09:20:54.612 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T09:20:54.612 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T09:20:54.612 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T09:20:54.612 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:20:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T09:20:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = True 2024-09-17T09:20:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T09:20:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:20:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T09:20:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T09:20:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T09:20:54.613 INFO:teuthology.orchestra.run.smithi148.stdout:Generating new minimal ceph.conf... 2024-09-17T09:20:55.643 INFO:teuthology.orchestra.run.smithi148.stdout:Restarting the monitor... 2024-09-17T09:20:55.658 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:55 smithi148 ceph-mon[25283]: from='client.? 172.21.15.148:0/938570563' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:20:55.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:55 smithi148 systemd[1]: Stopping Ceph mon.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:20:55.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:55 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a[25279]: 2024-09-17T09:20:55.901+0000 7f4711978640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:20:55.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:55 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a[25279]: 2024-09-17T09:20:55.901+0000 7f4711978640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-17T09:20:56.409 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:56 smithi148 podman[25527]: 2024-09-17 09:20:56.126293229 +0000 UTC m=+0.328519324 container died da3fa95be51e1f4c3e4b690d741044991e23df4d34554a522dfff419dd036f81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b) 2024-09-17T09:20:56.409 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:56 smithi148 podman[25527]: 2024-09-17 09:20:56.409086701 +0000 UTC m=+0.611312789 container cleanup da3fa95be51e1f4c3e4b690d741044991e23df4d34554a522dfff419dd036f81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-17T09:20:56.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:56 smithi148 bash[25527]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a 2024-09-17T09:20:56.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:56 smithi148 podman[25538]: 2024-09-17 09:20:56.651729881 +0000 UTC m=+0.517447811 container remove da3fa95be51e1f4c3e4b690d741044991e23df4d34554a522dfff419dd036f81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-17T09:20:57.076 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:56 smithi148 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-17T09:20:57.076 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:56 smithi148 systemd[1]: Stopped Ceph mon.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:20:57.076 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:56 smithi148 systemd[1]: Starting Ceph mon.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:20:57.076 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:56 smithi148 podman[25631]: 2024-09-17 09:20:56.977650464 +0000 UTC m=+0.108532434 container create b050b0b432b1c6651e3ae4517ed4c8331397fed3a0a4dbb5cc09bb8e769f6e73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-17T09:20:57.077 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:56 smithi148 podman[25631]: 2024-09-17 09:20:56.884942161 +0000 UTC m=+0.015824150 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T09:20:57.259 INFO:teuthology.orchestra.run.smithi148.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-17T09:20:57.378 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 podman[25631]: 2024-09-17 09:20:57.179872417 +0000 UTC m=+0.310754399 container init b050b0b432b1c6651e3ae4517ed4c8331397fed3a0a4dbb5cc09bb8e769f6e73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:20:57.378 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 podman[25631]: 2024-09-17 09:20:57.182981495 +0000 UTC m=+0.313863465 container start b050b0b432b1c6651e3ae4517ed4c8331397fed3a0a4dbb5cc09bb8e769f6e73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:20:57.378 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T09:20:57.378 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 2 2024-09-17T09:20:57.378 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: pidfile_write: ignore empty --pid-file 2024-09-17T09:20:57.378 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: load: jerasure load: lrc 2024-09-17T09:20:57.378 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: RocksDB version: 7.9.2 2024-09-17T09:20:57.378 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Git sha 0 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: DB SUMMARY 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: DB Session ID: 4P2SHBPDWPVELDRUF4O1 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: CURRENT file: CURRENT 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 83764 ; 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.error_if_exists: 0 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.create_if_missing: 0 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.paranoid_checks: 1 2024-09-17T09:20:57.379 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.env: 0x5582b164fbc0 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.info_log: 0x5582b2074be0 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.statistics: (nil) 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.use_fsync: 0 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_log_file_size: 0 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T09:20:57.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.allow_fallocate: 1 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.use_direct_reads: 0 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.db_log_dir: 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.wal_dir: 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T09:20:57.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.write_buffer_manager: 0x5582b2082320 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T09:20:57.382 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.unordered_write: 0 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.row_cache: None 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.wal_filter: None 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.two_write_queues: 0 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T09:20:57.383 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.wal_compression: 0 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.atomic_flush: 0 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.log_readahead_size: 0 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T09:20:57.384 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_background_jobs: 2 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_background_compactions: -1 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_subcompactions: 1 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_open_files: -1 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T09:20:57.385 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_background_flushes: -1 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Compression algorithms supported: 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: kZSTD supported: 0 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: kXpressCompression supported: 0 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: kBZip2Compression supported: 0 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: kLZ4Compression supported: 1 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: kZlibCompression supported: 1 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: kSnappyCompression supported: 1 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T09:20:57.386 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.merge_operator: 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_filter: None 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5582b20747e0) 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout: cache_index_and_filter_blocks: 1 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T09:20:57.387 INFO:journalctl@ceph.mon.a.smithi148.stdout: pin_top_level_index_and_filter: 1 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: index_type: 0 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: data_block_index_type: 0 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: index_shortening: 1 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: checksum: 4 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: no_block_cache: 0 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: block_cache: 0x5582b2069350 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: block_cache_name: BinnedLRUCache 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: block_cache_options: 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: capacity : 536870912 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: num_shard_bits : 4 2024-09-17T09:20:57.388 INFO:journalctl@ceph.mon.a.smithi148.stdout: strict_capacity_limit : 0 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: high_pri_pool_ratio: 0.000 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: block_cache_compressed: (nil) 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: persistent_cache: (nil) 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: block_size: 4096 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: block_size_deviation: 10 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: block_restart_interval: 16 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: index_block_restart_interval: 1 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: metadata_block_size: 4096 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: partition_filters: 0 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: use_delta_encoding: 1 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: filter_policy: bloomfilter 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: whole_key_filtering: 1 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: verify_compression: 0 2024-09-17T09:20:57.389 INFO:journalctl@ceph.mon.a.smithi148.stdout: read_amp_bytes_per_bit: 0 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout: format_version: 5 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout: enable_index_compression: 1 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout: block_align: 0 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout: max_auto_readahead_size: 262144 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout: prepopulate_block_cache: 0 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout: initial_auto_readahead_size: 8192 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compression: NoCompression 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T09:20:57.390 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.num_levels: 7 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T09:20:57.391 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T09:20:57.392 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T09:20:57.393 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T09:20:57.394 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.inplace_update_support: 0 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.bloom_locality: 0 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.max_successive_merges: 0 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.ttl: 2592000 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T09:20:57.395 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.enable_blob_files: false 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.min_blob_size: 0 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-17T09:20:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 39e1a2e6-51fa-40f1-8850-66dc07e0dcf4 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564857209669, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564857210995, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 80537, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 250, "table_properties": {"data_size": 78698, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9896, "raw_average_key_size": 47, "raw_value_size": 73006, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 209, "num_filter_entries": 209, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726564857, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "39e1a2e6-51fa-40f1-8850-66dc07e0dcf4", "db_session_id": "4P2SHBPDWPVELDRUF4O1", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564857211121, "job": 1, "event": "recovery_finished"} 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5582b2094e00 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: DB pointer 0x5582b2188000 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout: ** DB Stats ** 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T09:20:57.397 INFO:journalctl@ceph.mon.a.smithi148.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: ** Compaction Stats [default] ** 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: L0 2/0 80.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 89.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: Sum 2/0 80.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 89.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 89.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T09:20:57.398 INFO:journalctl@ceph.mon.a.smithi148.stdout: 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: ** Compaction Stats [default] ** 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 89.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T09:20:57.399 INFO:journalctl@ceph.mon.a.smithi148.stdout: Cumulative compaction: 0.00 GB write, 17.91 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout: Interval compaction: 0.00 GB write, 17.91 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout: Block cache BinnedLRUCache@0x5582b2069350#2 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout: 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: starting mon.a rank 0 at public addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] at bind addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: mon.a@-1(???) e1 preinit fsid ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: mon.a@-1(???).mds e1 new map 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: mon.a@-1(???).mds e1 print_map 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout: e1 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T09:20:57.400 INFO:journalctl@ceph.mon.a.smithi148.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout: legacy client fscid: -1 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout: 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout: No filesystems configured 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 bash[25631]: b050b0b432b1c6651e3ae4517ed4c8331397fed3a0a4dbb5cc09bb8e769f6e73 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 systemd[1]: Started Ceph mon.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: monmap e1: 1 mons at {a=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:20:57.401 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: fsmap 2024-09-17T09:20:57.402 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: osdmap e1: 0 total, 0 up, 0 in 2024-09-17T09:20:57.402 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:57 smithi148 ceph-mon[25650]: mgrmap e1: no daemons active 2024-09-17T09:20:58.356 INFO:teuthology.orchestra.run.smithi148.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-17T09:20:58.378 INFO:teuthology.orchestra.run.smithi148.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:20:58.379 INFO:teuthology.orchestra.run.smithi148.stdout:Creating mgr... 2024-09-17T09:20:58.379 INFO:teuthology.orchestra.run.smithi148.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-17T09:20:58.379 INFO:teuthology.orchestra.run.smithi148.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-17T09:20:58.646 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.y 2024-09-17T09:20:58.646 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-17T09:20:58.799 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ff0bb856-74d5-11ef-bceb-c7b262605968.target.wants/ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@.service. 2024-09-17T09:20:58.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:58 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/4072668303' entity='client.admin' 2024-09-17T09:20:58.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:58 smithi148 systemd[1]: Starting Ceph mgr.y for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:20:59.300 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 podman[25861]: 2024-09-17 09:20:58.926247381 +0000 UTC m=+0.015645855 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-17T09:20:59.300 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 podman[25861]: 2024-09-17 09:20:59.038726517 +0000 UTC m=+0.128124982 container create b26839806a720ed89c916e6345086d78727988d5b8a35253b6a9363275b363aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-17T09:20:59.300 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 podman[25861]: 2024-09-17 09:20:59.24081581 +0000 UTC m=+0.330214283 container init b26839806a720ed89c916e6345086d78727988d5b8a35253b6a9363275b363aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-17T09:20:59.300 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 podman[25861]: 2024-09-17 09:20:59.243720882 +0000 UTC m=+0.333119344 container start b26839806a720ed89c916e6345086d78727988d5b8a35253b6a9363275b363aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T09:20:59.300 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 bash[25861]: b26839806a720ed89c916e6345086d78727988d5b8a35253b6a9363275b363aa 2024-09-17T09:20:59.317 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T09:20:59.317 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout disabled 2024-09-17T09:20:59.322 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T09:20:59.322 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout inactive 2024-09-17T09:20:59.323 INFO:teuthology.orchestra.run.smithi148.stdout:firewalld.service is not enabled 2024-09-17T09:20:59.323 INFO:teuthology.orchestra.run.smithi148.stdout:Not possible to enable service . firewalld.service is not available 2024-09-17T09:20:59.331 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T09:20:59.331 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout disabled 2024-09-17T09:20:59.339 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T09:20:59.339 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout inactive 2024-09-17T09:20:59.339 INFO:teuthology.orchestra.run.smithi148.stdout:firewalld.service is not enabled 2024-09-17T09:20:59.339 INFO:teuthology.orchestra.run.smithi148.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-17T09:20:59.339 INFO:teuthology.orchestra.run.smithi148.stdout:Waiting for mgr to start... 2024-09-17T09:20:59.339 INFO:teuthology.orchestra.run.smithi148.stdout:Waiting for mgr... 2024-09-17T09:20:59.549 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 systemd[1]: Started Ceph mgr.y for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:20:59.550 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:20:59.368+0000 7f3c0b4f2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:20:59.550 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:20:59.429+0000 7f3c0b4f2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:20:59.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:20:59 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/1042395842' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T09:20:59.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:20:59.585+0000 7f3c0b4f2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:20:59.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:20:59.652+0000 7f3c0b4f2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:20:59.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:20:59.807+0000 7f3c0b4f2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:21:00.269 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:20:59 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:20:59.955+0000 7f3c0b4f2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:21:00.270 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:00.267+0000 7f3c0b4f2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:21:00.377 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "fsid": "ff0bb856-74d5-11ef-bceb-c7b262605968", 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 0 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T09:21:00.378 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T09:21:00.379 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T09:21:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T09:21:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T09:21:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T09:21:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T09:21:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T09:21:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T09:21:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T09:21:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T09:21:00.383 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T09:21:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T09:20:52.437675+0000", 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:00.385 INFO:teuthology.orchestra.run.smithi148.stdout:mgr not available, waiting (1/15)... 2024-09-17T09:21:00.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:00.330+0000 7f3c0b4f2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:21:00.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:00.512+0000 7f3c0b4f2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:21:01.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:01 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:01.376+0000 7f3c0b4f2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:21:01.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:01 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:01.442+0000 7f3c0b4f2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:21:02.276 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:02 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:02.160+0000 7f3c0b4f2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:21:02.276 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:02 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:02.274+0000 7f3c0b4f2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:21:02.529 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:02 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:02.423+0000 7f3c0b4f2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:21:02.529 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:02 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:02.485+0000 7f3c0b4f2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:21:02.796 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:02 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:02.663+0000 7f3c0b4f2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:21:02.796 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:02 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:02.732+0000 7f3c0b4f2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:21:02.796 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:02 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:02.793+0000 7f3c0b4f2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:21:03.092 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:02 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/915549381' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T09:21:03.092 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:02 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:02.959+0000 7f3c0b4f2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:21:03.092 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:03 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:03.023+0000 7f3c0b4f2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:21:03.092 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:03 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:03.089+0000 7f3c0b4f2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:21:03.420 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:03 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:03.173+0000 7f3c0b4f2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:21:03.420 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:03 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:03.232+0000 7f3c0b4f2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:21:03.421 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:03 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:03.291+0000 7f3c0b4f2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:21:03.571 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:03.571 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:03.571 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "fsid": "ff0bb856-74d5-11ef-bceb-c7b262605968", 2024-09-17T09:21:03.571 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T09:21:03.571 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 0 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-17T09:21:03.572 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:03.573 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T09:21:03.574 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:03.575 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T09:20:52.437675+0000", 2024-09-17T09:21:03.576 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:03.576 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:03.576 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T09:21:03.576 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:03.576 INFO:teuthology.orchestra.run.smithi148.stdout:mgr not available, waiting (2/15)... 2024-09-17T09:21:04.417 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:04.066+0000 7f3c0b4f2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:21:04.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: Activating manager daemon y 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: mgrmap e2: y(active, starting, since 0.00280213s) 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: from='mgr.14100 172.21.15.148:0/3698990912' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: from='mgr.14100 172.21.15.148:0/3698990912' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: from='mgr.14100 172.21.15.148:0/3698990912' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: from='mgr.14100 172.21.15.148:0/3698990912' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: from='mgr.14100 172.21.15.148:0/3698990912' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: Manager daemon y is now available 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: from='mgr.14100 172.21.15.148:0/3698990912' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: from='mgr.14100 172.21.15.148:0/3698990912' entity='mgr.y' 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: from='mgr.14100 172.21.15.148:0/3698990912' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: from='mgr.14100 172.21.15.148:0/3698990912' entity='mgr.y' 2024-09-17T09:21:04.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:04 smithi148 ceph-mon[25650]: from='mgr.14100 172.21.15.148:0/3698990912' entity='mgr.y' 2024-09-17T09:21:06.266 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:06 smithi148 ceph-mon[25650]: mgrmap e3: y(active, since 1.00653s) 2024-09-17T09:21:06.824 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:06.824 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:06.824 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "fsid": "ff0bb856-74d5-11ef-bceb-c7b262605968", 2024-09-17T09:21:06.824 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "health": { 2024-09-17T09:21:06.824 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-17T09:21:06.824 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-17T09:21:06.824 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 0 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "a" 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-17T09:21:06.825 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-17T09:21:06.826 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:06.826 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-17T09:21:06.826 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:06.826 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-17T09:21:06.826 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-17T09:21:06.826 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-17T09:21:06.826 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-17T09:21:06.829 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-17T09:21:06.829 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-17T09:21:06.829 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:06.829 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-17T09:21:06.830 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "restful" 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout ], 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "modified": "2024-09-17T09:20:52.437675+0000", 2024-09-17T09:21:06.831 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-17T09:21:06.832 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout }, 2024-09-17T09:21:06.832 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-17T09:21:06.832 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:06.832 INFO:teuthology.orchestra.run.smithi148.stdout:mgr is available 2024-09-17T09:21:07.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:07 smithi148 ceph-mon[25650]: mgrmap e4: y(active, since 2s) 2024-09-17T09:21:07.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:07 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/562523889' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-17T09:21:08.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:08 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3051617864' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-17T09:21:08.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:08 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3051617864' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-17T09:21:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout [global] 2024-09-17T09:21:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout fsid = ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:21:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-17T09:21:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-17T09:21:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-17T09:21:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-17T09:21:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:14.727 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-17T09:21:14.728 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-17T09:21:14.728 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 2024-09-17T09:21:14.728 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout [osd] 2024-09-17T09:21:14.728 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-17T09:21:14.728 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-17T09:21:14.728 INFO:teuthology.orchestra.run.smithi148.stdout:Enabling cephadm module... 2024-09-17T09:21:18.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:17 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/2944185326' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-17T09:21:18.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:17 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ignoring --setuser ceph since I am not root 2024-09-17T09:21:18.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:17 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ignoring --setgroup ceph since I am not root 2024-09-17T09:21:18.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:18 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:18.031+0000 7f672e924040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:21:18.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:18 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:18.088+0000 7f672e924040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:21:18.554 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:18 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:18.235+0000 7f672e924040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:21:18.554 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:18 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:18.301+0000 7f672e924040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:21:18.555 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:18 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:18.444+0000 7f672e924040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:21:18.805 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:18 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:18.588+0000 7f672e924040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:21:19.115 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:18 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/2944185326' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-17T09:21:19.115 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:18 smithi148 ceph-mon[25650]: mgrmap e5: y(active, since 13s) 2024-09-17T09:21:19.116 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:18 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:18.912+0000 7f672e924040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:21:19.116 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:18 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:18.974+0000 7f672e924040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:21:19.417 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:19 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:19.165+0000 7f672e924040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:21:19.888 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi148.stdout:Waiting for the mgr to restart... 2024-09-17T09:21:19.889 INFO:teuthology.orchestra.run.smithi148.stdout:Waiting for mgr epoch 5... 2024-09-17T09:21:20.008 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:19 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3535017548' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-17T09:21:20.313 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:20 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:20.059+0000 7f672e924040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:21:20.314 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:20 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:20.129+0000 7f672e924040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:21:21.102 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:20 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:20.846+0000 7f672e924040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:21:21.102 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:20 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:20.960+0000 7f672e924040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:21:21.375 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:21.100+0000 7f672e924040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:21:21.375 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:21.161+0000 7f672e924040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:21:21.376 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:21.306+0000 7f672e924040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:21:21.376 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:21.374+0000 7f672e924040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:21:21.656 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:21.434+0000 7f672e924040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:21:21.657 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:21.590+0000 7f672e924040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:21:21.657 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:21.655+0000 7f672e924040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:21:21.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:21.721+0000 7f672e924040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:21:21.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:21.803+0000 7f672e924040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:21:21.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:21.862+0000 7f672e924040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:21:22.417 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:21.920+0000 7f672e924040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:21:23.167 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:22.692+0000 7f672e924040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:21:23.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: Active manager daemon y restarted 2024-09-17T09:21:23.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: Activating manager daemon y 2024-09-17T09:21:23.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: osdmap e2: 0 total, 0 up, 0 in 2024-09-17T09:21:23.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: mgrmap e6: y(active, starting, since 0.00347704s) 2024-09-17T09:21:23.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:21:23.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:21:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:21:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:21:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:21:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: Manager daemon y is now available 2024-09-17T09:21:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:21:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:22 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:21:24.049 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:23 smithi148 ceph-mon[25650]: Found migration_current of "None". Setting to last migration. 2024-09-17T09:21:24.049 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:23 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:24.050 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:23 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:24.050 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:23 smithi148 ceph-mon[25650]: mgrmap e7: y(active, since 1.00492s) 2024-09-17T09:21:24.309 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:24.229+0000 7f669be4f640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:09:21:24] ENGINE Error in HTTPServer.serve 2024-09-17T09:21:24.309 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: Traceback (most recent call last): 2024-09-17T09:21:24.309 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:21:24.309 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self._connections.run(self.expiration_interval) 2024-09-17T09:21:24.309 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:21:24.309 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self._run(expiration_interval) 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: s = self.context.wrap_socket( 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: return self.sslsocket_class._create( 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self.do_handshake() 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self._sslobj.do_handshake() 2024-09-17T09:21:24.310 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:21:24.311 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:24.311 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:24.311 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-17T09:21:24.311 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T09:21:24.311 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:24.312 INFO:teuthology.orchestra.run.smithi148.stdout:mgr epoch 5 is available 2024-09-17T09:21:24.312 INFO:teuthology.orchestra.run.smithi148.stdout:Setting orchestrator backend to cephadm... 2024-09-17T09:21:24.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-mon[25650]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-17T09:21:24.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-mon[25650]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-17T09:21:24.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:24 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:26.003 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:25 smithi148 ceph-mon[25650]: [17/Sep/2024:09:21:24] ENGINE Bus STARTING 2024-09-17T09:21:26.003 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:25 smithi148 ceph-mon[25650]: [17/Sep/2024:09:21:24] ENGINE Serving on https://172.21.15.148:7150 2024-09-17T09:21:26.003 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:25 smithi148 ceph-mon[25650]: [17/Sep/2024:09:21:24] ENGINE Error in HTTPServer.serve 2024-09-17T09:21:26.003 INFO:journalctl@ceph.mon.a.smithi148.stdout: Traceback (most recent call last): 2024-09-17T09:21:26.003 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:21:26.003 INFO:journalctl@ceph.mon.a.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-09-17T09:21:26.003 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:21:26.003 INFO:journalctl@ceph.mon.a.smithi148.stdout: self._run(expiration_interval) 2024-09-17T09:21:26.003 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:21:26.003 INFO:journalctl@ceph.mon.a.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:21:26.003 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout: s = self.context.wrap_socket( 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout: return self.sslsocket_class._create( 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout: self.do_handshake() 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout: self._sslobj.do_handshake() 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:25 smithi148 ceph-mon[25650]: [17/Sep/2024:09:21:24] ENGINE Serving on http://172.21.15.148:8765 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:25 smithi148 ceph-mon[25650]: [17/Sep/2024:09:21:24] ENGINE Bus STARTED 2024-09-17T09:21:26.004 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:25 smithi148 ceph-mon[25650]: mgrmap e8: y(active, since 2s) 2024-09-17T09:21:26.005 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:25 smithi148 ceph-mon[25650]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:26.005 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:25 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:26.005 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:25 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:26.540 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-17T09:21:26.540 INFO:teuthology.orchestra.run.smithi148.stdout:Generating ssh key... 2024-09-17T09:21:27.182 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:26 smithi148 ceph-mon[25650]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:27.618 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: Generating public/private rsa key pair. 2024-09-17T09:21:27.618 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: Your identification has been saved in /tmp/tmpby9z143e/key 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: Your public key has been saved in /tmp/tmpby9z143e/key.pub 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: The key fingerprint is: 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: SHA256:MAtxYdUNHdbDfj5r6ZjjpOkydQmu6yLImNEoxulEBOs ceph-ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: The key's randomart image is: 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: +---[RSA 3072]----+ 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: |o . +o...+o+ | 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: | o + ..o + | 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: |o . o . . | 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: |.. . + . . .| 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: |oE+ . S . . .o | 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: |.B . o o ..| 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: |= = . o .. +| 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: | + o . . + +.o+ | 2024-09-17T09:21:27.619 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: | . oo=+.++. | 2024-09-17T09:21:27.620 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: +----[SHA256]-----+ 2024-09-17T09:21:28.457 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:28 smithi148 ceph-mon[25650]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:28.457 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:28 smithi148 ceph-mon[25650]: Generating ssh key... 2024-09-17T09:21:28.457 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:28 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:28.457 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:28 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:28.693 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCMpBNfKup/N3EgZPJ4K4u2LXb16knkzmLPjvXdltIiLGaUOvuQVBWhq5FQew8VSyi/lhBSQTHiY7iAuCMmJmh2D57EQh0N0eOKpV2nzGHspmeOlokZWt780HdYccJQVpy9wpvHlVT7SQ1qjPSTsObou40WQvXdFWq11JKoH5LOTM10oSZQcqWwLnniBrvJO9NZtTloHtVZsTtVLVWLLXXl5sJqJGZK1T3joJzlqvVjdkulyomKf1A3oTx8UsqVkgamDVZezYlljakmNh5NKMoHwKrwY/rgCn1O8Ujjgu97JEqbP6EMWGImyU3I421Rzd8rvGpdl5C1+u+hiPP1WvXslh8F8O7H3DI+XhCKLV0h0CJHm7hXTEBjdGgU6FhAMi+WKnGQyPqJOEeVAlqIADdvvTYWTJax6hkrC//9Lgpk9yTezI19326pOtjNapJkWY3j75GZDTkyz4N7cCiklOBplAWtih8CLZVIoXhQhKzi3bxhQeoHxWOYmbXfqp0LxP8= ceph-ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:21:28.693 INFO:teuthology.orchestra.run.smithi148.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-17T09:21:28.693 INFO:teuthology.orchestra.run.smithi148.stdout:Adding key to root@localhost authorized_keys... 2024-09-17T09:21:28.705 INFO:teuthology.orchestra.run.smithi148.stdout:Adding host smithi148... 2024-09-17T09:21:29.484 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:29 smithi148 ceph-mon[25650]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:30.271 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:30 smithi148 ceph-mon[25650]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "addr": "172.21.15.148", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:31.344 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:31 smithi148 ceph-mon[25650]: Deploying cephadm binary to smithi148 2024-09-17T09:21:32.418 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout Added host 'smithi148' with addr '172.21.15.148' 2024-09-17T09:21:32.418 INFO:teuthology.orchestra.run.smithi148.stdout:Deploying unmanaged mon service... 2024-09-17T09:21:32.977 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:32 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:32.977 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:32 smithi148 ceph-mon[25650]: Added host smithi148 2024-09-17T09:21:32.977 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:32 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:33.608 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-17T09:21:33.608 INFO:teuthology.orchestra.run.smithi148.stdout:Deploying unmanaged mgr service... 2024-09-17T09:21:34.252 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:33 smithi148 ceph-mon[25650]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:34.253 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:33 smithi148 ceph-mon[25650]: Saving service mon spec with placement count:5 2024-09-17T09:21:34.253 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:33 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:35.336 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-17T09:21:35.636 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:35 smithi148 ceph-mon[25650]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:35.637 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:35 smithi148 ceph-mon[25650]: Saving service mgr spec with placement count:2 2024-09-17T09:21:35.637 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:35 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:35.637 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:35 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:35.637 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:35 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:37.156 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:36 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/2994334067' entity='client.admin' 2024-09-17T09:21:37.157 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:36 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:37.157 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:36 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:37.157 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:36 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:21:37.157 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:36 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' 2024-09-17T09:21:37.157 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:36 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi148", "caps": []}]: dispatch 2024-09-17T09:21:37.157 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:36 smithi148 ceph-mon[25650]: from='mgr.14118 172.21.15.148:0/2796702981' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi148", "caps": []}]': finished 2024-09-17T09:21:38.268 INFO:teuthology.orchestra.run.smithi148.stdout:Enabling the dashboard module... 2024-09-17T09:21:38.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:38 smithi148 ceph-mon[25650]: Deploying daemon agent.smithi148 on smithi148 2024-09-17T09:21:38.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:38 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/220928202' entity='client.admin' 2024-09-17T09:21:39.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:39 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3829984420' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-17T09:21:39.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ignoring --setuser ceph since I am not root 2024-09-17T09:21:39.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ignoring --setgroup ceph since I am not root 2024-09-17T09:21:39.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:39.815+0000 7f2ceb075040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:21:39.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:39.872+0000 7f2ceb075040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:21:40.280 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:40 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:40.020+0000 7f2ceb075040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:21:40.281 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:40 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:40.086+0000 7f2ceb075040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:21:40.281 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:40 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:40.232+0000 7f2ceb075040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:21:40.548 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:40 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:40.376+0000 7f2ceb075040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:21:40.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:40 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3829984420' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-17T09:21:40.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:40 smithi148 ceph-mon[25650]: mgrmap e9: y(active, since 16s) 2024-09-17T09:21:40.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:40 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:40.689+0000 7f2ceb075040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:21:40.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:40 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:40.753+0000 7f2ceb075040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:21:41.238 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:40 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:40.942+0000 7f2ceb075040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:21:41.525 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:41.526 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-17T09:21:41.526 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-17T09:21:41.526 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-17T09:21:41.526 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-17T09:21:41.526 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:41.526 INFO:teuthology.orchestra.run.smithi148.stdout:Waiting for the mgr to restart... 2024-09-17T09:21:41.526 INFO:teuthology.orchestra.run.smithi148.stdout:Waiting for mgr epoch 9... 2024-09-17T09:21:41.835 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:41 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3372987835' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-17T09:21:42.167 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:41 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:41.833+0000 7f2ceb075040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:21:42.167 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:41 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:41.903+0000 7f2ceb075040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:21:42.863 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:42.607+0000 7f2ceb075040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:21:42.863 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:42.721+0000 7f2ceb075040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:21:43.133 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:42.860+0000 7f2ceb075040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:21:43.133 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:42.920+0000 7f2ceb075040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:21:43.133 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:43.064+0000 7f2ceb075040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:21:43.134 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:43.130+0000 7f2ceb075040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:21:43.410 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:43.191+0000 7f2ceb075040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:21:43.410 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:43.355+0000 7f2ceb075040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:21:43.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:43.424+0000 7f2ceb075040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:21:43.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:43.493+0000 7f2ceb075040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:21:43.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:43.581+0000 7f2ceb075040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:21:43.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:43.644+0000 7f2ceb075040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:21:44.167 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:43.706+0000 7f2ceb075040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:21:44.650 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:44.493+0000 7f2ceb075040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:21:44.650 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: Active manager daemon y restarted 2024-09-17T09:21:44.650 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: Activating manager daemon y 2024-09-17T09:21:44.650 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: osdmap e3: 0 total, 0 up, 0 in 2024-09-17T09:21:44.650 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: mgrmap e10: y(active, starting, since 0.00327667s) 2024-09-17T09:21:44.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:21:44.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:21:44.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:21:44.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:21:44.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:21:44.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: Manager daemon y is now available 2024-09-17T09:21:44.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:44.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:44.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:21:44.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:21:45.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:45.581+0000 7f2c514ff640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:09:21:45] ENGINE Error in HTTPServer.serve 2024-09-17T09:21:45.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: Traceback (most recent call last): 2024-09-17T09:21:45.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:21:45.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self._connections.run(self.expiration_interval) 2024-09-17T09:21:45.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:21:45.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self._run(expiration_interval) 2024-09-17T09:21:45.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:21:45.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:21:45.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:21:45.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:21:45.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:21:45.918 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: s = self.context.wrap_socket( 2024-09-17T09:21:45.919 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:21:45.919 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: return self.sslsocket_class._create( 2024-09-17T09:21:45.919 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:21:45.919 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self.do_handshake() 2024-09-17T09:21:45.919 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:21:45.919 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self._sslobj.do_handshake() 2024-09-17T09:21:45.919 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:21:45.919 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:21:45 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:21:46.226 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout { 2024-09-17T09:21:46.226 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-17T09:21:46.226 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-17T09:21:46.226 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout } 2024-09-17T09:21:46.226 INFO:teuthology.orchestra.run.smithi148.stdout:mgr epoch 9 is available 2024-09-17T09:21:46.226 INFO:teuthology.orchestra.run.smithi148.stdout:Generating a dashboard self-signed certificate... 2024-09-17T09:21:46.907 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: [17/Sep/2024:09:21:45] ENGINE Bus STARTING 2024-09-17T09:21:46.907 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: [17/Sep/2024:09:21:45] ENGINE Serving on http://172.21.15.148:8765 2024-09-17T09:21:46.907 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-17T09:21:46.907 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: mgrmap e11: y(active, since 1.00548s) 2024-09-17T09:21:46.907 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-17T09:21:46.907 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: [17/Sep/2024:09:21:45] ENGINE Serving on https://172.21.15.148:7150 2024-09-17T09:21:46.907 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: [17/Sep/2024:09:21:45] ENGINE Bus STARTED 2024-09-17T09:21:46.907 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: [17/Sep/2024:09:21:45] ENGINE Error in HTTPServer.serve 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: Traceback (most recent call last): 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: self._run(expiration_interval) 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: s = self.context.wrap_socket( 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:21:46.908 INFO:journalctl@ceph.mon.a.smithi148.stdout: return self.sslsocket_class._create( 2024-09-17T09:21:46.909 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:21:46.909 INFO:journalctl@ceph.mon.a.smithi148.stdout: self.do_handshake() 2024-09-17T09:21:46.909 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:21:46.909 INFO:journalctl@ceph.mon.a.smithi148.stdout: self._sslobj.do_handshake() 2024-09-17T09:21:46.909 INFO:journalctl@ceph.mon.a.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:21:46.909 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:46.909 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:21:46.909 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:46.909 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi148", "caps": []}]: dispatch 2024-09-17T09:21:47.515 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-17T09:21:47.515 INFO:teuthology.orchestra.run.smithi148.stdout:Creating initial admin user... 2024-09-17T09:21:48.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:47 smithi148 ceph-mon[25650]: Deploying daemon agent.smithi148 on smithi148 2024-09-17T09:21:48.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:47 smithi148 ceph-mon[25650]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:48.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:47 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:48.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:47 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:48.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:47 smithi148 ceph-mon[25650]: mgrmap e12: y(active, since 2s) 2024-09-17T09:21:48.905 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$NSxj9EkdD0EqyvIIFBza2ei6FWXzUART4Itb6Hl8lPnDbHnCX51Qe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726564908, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-17T09:21:48.905 INFO:teuthology.orchestra.run.smithi148.stdout:Fetching dashboard port number... 2024-09-17T09:21:49.482 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:49 smithi148 ceph-mon[25650]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:21:49.482 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:49 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:50.019 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stdout 8443 2024-09-17T09:21:50.028 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-17T09:21:50.028 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout disabled 2024-09-17T09:21:50.035 INFO:teuthology.orchestra.run.smithi148.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-17T09:21:50.035 INFO:teuthology.orchestra.run.smithi148.stdout:systemctl: stdout inactive 2024-09-17T09:21:50.036 INFO:teuthology.orchestra.run.smithi148.stdout:firewalld.service is not enabled 2024-09-17T09:21:50.036 INFO:teuthology.orchestra.run.smithi148.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-17T09:21:50.037 INFO:teuthology.orchestra.run.smithi148.stdout:Ceph Dashboard is now available at: 2024-09-17T09:21:50.038 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:21:50.038 INFO:teuthology.orchestra.run.smithi148.stdout: URL: https://smithi148.front.sepia.ceph.com:8443/ 2024-09-17T09:21:50.038 INFO:teuthology.orchestra.run.smithi148.stdout: User: admin 2024-09-17T09:21:50.038 INFO:teuthology.orchestra.run.smithi148.stdout: Password: cb7q9i7wjd 2024-09-17T09:21:50.038 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:21:50.038 INFO:teuthology.orchestra.run.smithi148.stdout:Saving cluster configuration to /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config directory 2024-09-17T09:21:50.038 INFO:teuthology.orchestra.run.smithi148.stdout:Enabling autotune for osd_memory_target 2024-09-17T09:21:50.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:50 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/2488799040' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-17T09:21:52.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:52 smithi148 ceph-mon[25650]: mgrmap e13: y(active, since 6s) 2024-09-17T09:21:52.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:52 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3745412739' entity='client.admin' 2024-09-17T09:21:52.424 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-17T09:21:52.424 INFO:teuthology.orchestra.run.smithi148.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-17T09:21:52.424 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:21:52.424 INFO:teuthology.orchestra.run.smithi148.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:21:52.424 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:21:52.424 INFO:teuthology.orchestra.run.smithi148.stdout:Or, if you are only running a single cluster on this host: 2024-09-17T09:21:52.424 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:21:52.425 INFO:teuthology.orchestra.run.smithi148.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-17T09:21:52.425 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:21:52.425 INFO:teuthology.orchestra.run.smithi148.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-17T09:21:52.425 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:21:52.425 INFO:teuthology.orchestra.run.smithi148.stdout: ceph telemetry on 2024-09-17T09:21:52.425 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:21:52.425 INFO:teuthology.orchestra.run.smithi148.stdout:For more information see: 2024-09-17T09:21:52.425 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:21:52.425 INFO:teuthology.orchestra.run.smithi148.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-17T09:21:52.425 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:21:52.425 INFO:teuthology.orchestra.run.smithi148.stdout:Bootstrap complete. 2024-09-17T09:21:52.457 INFO:tasks.cephadm:Fetching config... 2024-09-17T09:21:52.458 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:21:52.458 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-17T09:21:52.474 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-17T09:21:52.474 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:21:52.475 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-17T09:21:52.530 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-17T09:21:52.530 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:21:52.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-17T09:21:52.682 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-17T09:21:52.682 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:21:52.682 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-17T09:21:52.737 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-17T09:21:52.737 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCMpBNfKup/N3EgZPJ4K4u2LXb16knkzmLPjvXdltIiLGaUOvuQVBWhq5FQew8VSyi/lhBSQTHiY7iAuCMmJmh2D57EQh0N0eOKpV2nzGHspmeOlokZWt780HdYccJQVpy9wpvHlVT7SQ1qjPSTsObou40WQvXdFWq11JKoH5LOTM10oSZQcqWwLnniBrvJO9NZtTloHtVZsTtVLVWLLXXl5sJqJGZK1T3joJzlqvVjdkulyomKf1A3oTx8UsqVkgamDVZezYlljakmNh5NKMoHwKrwY/rgCn1O8Ujjgu97JEqbP6EMWGImyU3I421Rzd8rvGpdl5C1+u+hiPP1WvXslh8F8O7H3DI+XhCKLV0h0CJHm7hXTEBjdGgU6FhAMi+WKnGQyPqJOEeVAlqIADdvvTYWTJax6hkrC//9Lgpk9yTezI19326pOtjNapJkWY3j75GZDTkyz4N7cCiklOBplAWtih8CLZVIoXhQhKzi3bxhQeoHxWOYmbXfqp0LxP8= ceph-ff0bb856-74d5-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T09:21:52.893 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCMpBNfKup/N3EgZPJ4K4u2LXb16knkzmLPjvXdltIiLGaUOvuQVBWhq5FQew8VSyi/lhBSQTHiY7iAuCMmJmh2D57EQh0N0eOKpV2nzGHspmeOlokZWt780HdYccJQVpy9wpvHlVT7SQ1qjPSTsObou40WQvXdFWq11JKoH5LOTM10oSZQcqWwLnniBrvJO9NZtTloHtVZsTtVLVWLLXXl5sJqJGZK1T3joJzlqvVjdkulyomKf1A3oTx8UsqVkgamDVZezYlljakmNh5NKMoHwKrwY/rgCn1O8Ujjgu97JEqbP6EMWGImyU3I421Rzd8rvGpdl5C1+u+hiPP1WvXslh8F8O7H3DI+XhCKLV0h0CJHm7hXTEBjdGgU6FhAMi+WKnGQyPqJOEeVAlqIADdvvTYWTJax6hkrC//9Lgpk9yTezI19326pOtjNapJkWY3j75GZDTkyz4N7cCiklOBplAWtih8CLZVIoXhQhKzi3bxhQeoHxWOYmbXfqp0LxP8= ceph-ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:21:52.905 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCMpBNfKup/N3EgZPJ4K4u2LXb16knkzmLPjvXdltIiLGaUOvuQVBWhq5FQew8VSyi/lhBSQTHiY7iAuCMmJmh2D57EQh0N0eOKpV2nzGHspmeOlokZWt780HdYccJQVpy9wpvHlVT7SQ1qjPSTsObou40WQvXdFWq11JKoH5LOTM10oSZQcqWwLnniBrvJO9NZtTloHtVZsTtVLVWLLXXl5sJqJGZK1T3joJzlqvVjdkulyomKf1A3oTx8UsqVkgamDVZezYlljakmNh5NKMoHwKrwY/rgCn1O8Ujjgu97JEqbP6EMWGImyU3I421Rzd8rvGpdl5C1+u+hiPP1WvXslh8F8O7H3DI+XhCKLV0h0CJHm7hXTEBjdGgU6FhAMi+WKnGQyPqJOEeVAlqIADdvvTYWTJax6hkrC//9Lgpk9yTezI19326pOtjNapJkWY3j75GZDTkyz4N7cCiklOBplAWtih8CLZVIoXhQhKzi3bxhQeoHxWOYmbXfqp0LxP8= ceph-ff0bb856-74d5-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-17T09:21:53.054 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCMpBNfKup/N3EgZPJ4K4u2LXb16knkzmLPjvXdltIiLGaUOvuQVBWhq5FQew8VSyi/lhBSQTHiY7iAuCMmJmh2D57EQh0N0eOKpV2nzGHspmeOlokZWt780HdYccJQVpy9wpvHlVT7SQ1qjPSTsObou40WQvXdFWq11JKoH5LOTM10oSZQcqWwLnniBrvJO9NZtTloHtVZsTtVLVWLLXXl5sJqJGZK1T3joJzlqvVjdkulyomKf1A3oTx8UsqVkgamDVZezYlljakmNh5NKMoHwKrwY/rgCn1O8Ujjgu97JEqbP6EMWGImyU3I421Rzd8rvGpdl5C1+u+hiPP1WvXslh8F8O7H3DI+XhCKLV0h0CJHm7hXTEBjdGgU6FhAMi+WKnGQyPqJOEeVAlqIADdvvTYWTJax6hkrC//9Lgpk9yTezI19326pOtjNapJkWY3j75GZDTkyz4N7cCiklOBplAWtih8CLZVIoXhQhKzi3bxhQeoHxWOYmbXfqp0LxP8= ceph-ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:21:53.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-17T09:21:53.351 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:21:54.511 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:54.511 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:54.511 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:54.511 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:54.511 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:54.511 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:54.511 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:54.512 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:54.512 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:54.512 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:54.512 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:54.512 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:55.653 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:55 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:55.653 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:55 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/4228399295' entity='client.admin' 2024-09-17T09:21:55.653 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:55 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:21:55.653 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:21:55 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:21:55.674 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-17T09:21:55.674 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-17T09:21:56.340 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:00.219 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2024-09-17T09:22:00.220 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:22:00.220 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2024-09-17T09:22:00.239 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:22:00.239 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:00.297 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2024-09-17T09:22:00.297 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch host add smithi157 2024-09-17T09:22:00.584 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:00 smithi148 ceph-mon[25650]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:00.584 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:00.585 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:00.585 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:00.585 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:00.585 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:00 smithi148 ceph-mon[25650]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-17T09:22:00.585 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:00 smithi148 ceph-mon[25650]: Updating smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:00.585 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:00 smithi148 ceph-mon[25650]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:00.585 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:00.585 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:00.673 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:01.913 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:01 smithi148 ceph-mon[25650]: Updating smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:22:01.913 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:01.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:01.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:01.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:01.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:03.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:02 smithi148 ceph-mon[25650]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:04.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:03 smithi148 ceph-mon[25650]: Deploying cephadm binary to smithi157 2024-09-17T09:22:04.934 INFO:teuthology.orchestra.run.smithi148.stdout:Added host 'smithi157' with addr '172.21.15.157' 2024-09-17T09:22:05.554 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:05 smithi148 ceph-mon[25650]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:05.554 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:05.555 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:05.555 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:05.555 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:05.555 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:05.555 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:22:05.555 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:05.555 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:05.576 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-17T09:22:05.770 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:06.862 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:06 smithi148 ceph-mon[25650]: Added host smithi157 2024-09-17T09:22:06.862 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:06 smithi148 ceph-mon[25650]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-17T09:22:06.862 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:06 smithi148 ceph-mon[25650]: Updating smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:06.862 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:06.862 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:06.862 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi157", "caps": []}]: dispatch 2024-09-17T09:22:06.862 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi157", "caps": []}]': finished 2024-09-17T09:22:07.528 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:22:07.529 INFO:teuthology.orchestra.run.smithi148.stdout:[{"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2024-09-17T09:22:07.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:07.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: Updating smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:22:07.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:07.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:07.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:07.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:07.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:07.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:07.828 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:07.828 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:07.828 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:07.828 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:07.828 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:07.828 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:08.115 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-17T09:22:08.115 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-17T09:22:08.303 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:08.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:08 smithi148 ceph-mon[25650]: Deploying daemon agent.smithi157 on smithi157 2024-09-17T09:22:08.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:08 smithi148 ceph-mon[25650]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:22:08.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:08 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:08.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:08 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:08.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:08 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:09.635 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:09 smithi148 ceph-mon[25650]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:09.635 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:09 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:10.606 INFO:teuthology.orchestra.run.smithi148.stderr:adjusted tunables profile to default 2024-09-17T09:22:10.897 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:10 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3998140238' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-17T09:22:10.897 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:10.897 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:11.110 INFO:tasks.cephadm:Adding mon.a on smithi148 2024-09-17T09:22:11.110 INFO:tasks.cephadm:Adding mon.c on smithi148 2024-09-17T09:22:11.110 INFO:tasks.cephadm:Adding mon.b on smithi157 2024-09-17T09:22:11.110 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi148:172.21.15.148=a;smithi148:[v2:172.21.15.148:3301,v1:172.21.15.148:6790]=c;smithi157:172.21.15.157=b' 2024-09-17T09:22:11.308 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:11.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:11 smithi148 ceph-mon[25650]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:11.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:11 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3998140238' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T09:22:11.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:11 smithi148 ceph-mon[25650]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:22:12.822 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2024-09-17T09:22:13.395 DEBUG:teuthology.orchestra.run.smithi148:mon.c> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.c.service 2024-09-17T09:22:13.397 DEBUG:teuthology.orchestra.run.smithi157:mon.b> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.b.service 2024-09-17T09:22:13.399 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T09:22:13.400 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T09:22:13.719 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:13 smithi148 ceph-mon[25650]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:13.720 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:13.720 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:13.720 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:13.720 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:13.720 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:13.720 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:13.720 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:14.520 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:14.900 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:14 smithi148 ceph-mon[25650]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi148:172.21.15.148=a;smithi148:[v2:172.21.15.148:3301,v1:172.21.15.148:6790]=c;smithi157:172.21.15.157=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:14.900 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:14 smithi148 ceph-mon[25650]: Saving service mon spec with placement smithi148:172.21.15.148=a;smithi148:[v2:172.21.15.148:3301,v1:172.21.15.148:6790]=c;smithi157:172.21.15.157=b;count:3 2024-09-17T09:22:14.900 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:14 smithi148 ceph-mon[25650]: Deploying daemon mon.b on smithi157 2024-09-17T09:22:15.812 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:15 smithi148 ceph-mon[25650]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:15.812 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:15.813 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:15.813 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:17.217 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:17 smithi157 systemd[1]: Starting Ceph mon.b for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:22:17.418 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:22:17.419 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"ff0bb856-74d5-11ef-bceb-c7b262605968","modified":"2024-09-17T09:20:49.437650Z","created":"2024-09-17T09:20:49.437650Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-17T09:22:17.419 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-09-17T09:22:17.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:17 smithi157 podman[27366]: 2024-09-17 09:22:17.333189868 +0000 UTC m=+0.015521689 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:22:17.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:17 smithi157 podman[27366]: 2024-09-17 09:22:17.431643359 +0000 UTC m=+0.113975180 container create 58bf671c333fea30bd71c4f379b44fcffcf8d7f8b331ac1fa6311341f8186f5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=reef-19e2a2b, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T09:22:17.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:17 smithi148 ceph-mon[25650]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:17.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:17 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/3722377706' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:22:18.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 podman[27366]: 2024-09-17 09:22:18.187704826 +0000 UTC m=+0.870036658 container init 58bf671c333fea30bd71c4f379b44fcffcf8d7f8b331ac1fa6311341f8186f5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-17T09:22:18.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 podman[27366]: 2024-09-17 09:22:18.191252597 +0000 UTC m=+0.873584421 container start 58bf671c333fea30bd71c4f379b44fcffcf8d7f8b331ac1fa6311341f8186f5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-17T09:22:18.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: set uid:gid to 167:167 (ceph:ceph) 2024-09-17T09:22:18.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: ceph version 18.2.4-855-g19e2a2be (19e2a2beb6f41a3ab62650e38614eb29b59dcbb9) reef (stable), process ceph-mon, pid 2 2024-09-17T09:22:18.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: pidfile_write: ignore empty --pid-file 2024-09-17T09:22:18.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: load: jerasure load: lrc 2024-09-17T09:22:18.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: RocksDB version: 7.9.2 2024-09-17T09:22:18.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Git sha 0 2024-09-17T09:22:18.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Compile date 2024-09-16 15:15:13 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: DB SUMMARY 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: DB Session ID: R5V4D74C79HWJLQE72QC 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: CURRENT file: CURRENT 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: IDENTITY file: IDENTITY 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.error_if_exists: 0 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.create_if_missing: 0 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.paranoid_checks: 1 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-17T09:22:18.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-17T09:22:18.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-17T09:22:18.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.env: 0x55cf696bebc0 2024-09-17T09:22:18.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.fs: PosixFileSystem 2024-09-17T09:22:18.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.info_log: 0x55cf6b4a8be0 2024-09-17T09:22:18.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_file_opening_threads: 16 2024-09-17T09:22:18.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.statistics: (nil) 2024-09-17T09:22:18.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.use_fsync: 0 2024-09-17T09:22:18.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_log_file_size: 0 2024-09-17T09:22:18.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-17T09:22:18.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-17T09:22:18.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.keep_log_file_num: 1000 2024-09-17T09:22:18.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.recycle_log_file_num: 0 2024-09-17T09:22:18.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.allow_fallocate: 1 2024-09-17T09:22:18.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.allow_mmap_reads: 0 2024-09-17T09:22:18.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.allow_mmap_writes: 0 2024-09-17T09:22:18.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.use_direct_reads: 0 2024-09-17T09:22:18.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-17T09:22:18.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.create_missing_column_families: 0 2024-09-17T09:22:18.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.db_log_dir: 2024-09-17T09:22:18.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.wal_dir: 2024-09-17T09:22:18.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-17T09:22:18.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-17T09:22:18.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-17T09:22:18.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-17T09:22:18.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-17T09:22:18.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-17T09:22:18.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.advise_random_on_open: 1 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.db_write_buffer_size: 0 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.write_buffer_manager: 0x55cf6b4b6320 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.rate_limiter: (nil) 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.wal_recovery_mode: 2 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.enable_thread_tracking: 0 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.enable_pipelined_write: 0 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.unordered_write: 0 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-17T09:22:18.277 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.row_cache: None 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.wal_filter: None 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.allow_ingest_behind: 0 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.two_write_queues: 0 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.manual_wal_flush: 0 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.wal_compression: 0 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.atomic_flush: 0 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.log_readahead_size: 0 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-17T09:22:18.278 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.best_efforts_recovery: 0 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.allow_data_in_errors: 0 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.db_host_id: __hostname__ 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_background_jobs: 2 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_background_compactions: -1 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_subcompactions: 1 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_total_wal_size: 0 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-17T09:22:18.279 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_open_files: -1 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bytes_per_sync: 0 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_readahead_size: 0 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_background_flushes: -1 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Compression algorithms supported: 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: kZSTD supported: 0 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: kXpressCompression supported: 0 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: kBZip2Compression supported: 0 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-17T09:22:18.280 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: kLZ4Compression supported: 1 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: kZlibCompression supported: 1 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: kLZ4HCCompression supported: 1 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: kSnappyCompression supported: 1 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.merge_operator: 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_filter: None 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_filter_factory: None 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.sst_partitioner_factory: None 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-17T09:22:18.281 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cf6b4a87e0) 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: cache_index_and_filter_blocks: 1 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: pin_top_level_index_and_filter: 1 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: index_type: 0 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: data_block_index_type: 0 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: index_shortening: 1 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: checksum: 4 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: no_block_cache: 0 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_cache: 0x55cf6b49d350 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_cache_name: BinnedLRUCache 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_cache_options: 2024-09-17T09:22:18.282 INFO:journalctl@ceph.mon.b.smithi157.stdout: capacity : 536870912 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: num_shard_bits : 4 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: strict_capacity_limit : 0 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: high_pri_pool_ratio: 0.000 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_cache_compressed: (nil) 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: persistent_cache: (nil) 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_size: 4096 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_size_deviation: 10 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_restart_interval: 16 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: index_block_restart_interval: 1 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: metadata_block_size: 4096 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: partition_filters: 0 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: use_delta_encoding: 1 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: filter_policy: bloomfilter 2024-09-17T09:22:18.283 INFO:journalctl@ceph.mon.b.smithi157.stdout: whole_key_filtering: 1 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout: verify_compression: 0 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout: read_amp_bytes_per_bit: 0 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout: format_version: 5 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout: enable_index_compression: 1 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout: block_align: 0 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout: max_auto_readahead_size: 262144 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout: prepopulate_block_cache: 0 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout: initial_auto_readahead_size: 8192 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout: num_file_reads_for_auto_readahead: 2 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.write_buffer_size: 33554432 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_write_buffer_number: 2 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compression: NoCompression 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bottommost_compression: Disabled 2024-09-17T09:22:18.284 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.prefix_extractor: nullptr 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.num_levels: 7 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-17T09:22:18.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compression_opts.level: 32767 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compression_opts.strategy: 0 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compression_opts.enabled: false 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-17T09:22:18.286 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.target_file_size_base: 67108864 2024-09-17T09:22:18.287 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-17T09:22:18.287 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-17T09:22:18.287 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-17T09:22:18.287 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-17T09:22:18.287 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-17T09:22:18.287 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-17T09:22:18.288 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-17T09:22:18.288 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-17T09:22:18.288 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-17T09:22:18.288 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-17T09:22:18.288 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-17T09:22:18.288 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.arena_block_size: 1048576 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.disable_auto_compactions: 0 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-17T09:22:18.289 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.inplace_update_support: 0 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.bloom_locality: 0 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.max_successive_merges: 0 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-17T09:22:18.290 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.paranoid_file_checks: 0 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.force_consistency_checks: 1 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.report_bg_io_stats: 0 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.ttl: 2592000 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.enable_blob_files: false 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.min_blob_size: 0 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.blob_file_size: 268435456 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-17T09:22:18.291 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.blob_file_starting_level: 0 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f91185f1-8061-4c98-8172-a3a58eb12c66 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564938220022, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564938220423, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726564938, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f91185f1-8061-4c98-8172-a3a58eb12c66", "db_session_id": "R5V4D74C79HWJLQE72QC", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726564938220523, "job": 1, "event": "recovery_finished"} 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cf6b4c8e00 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: DB pointer 0x55cf6b5bc000 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-17T09:22:18.292 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: ** DB Stats ** 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: ** Compaction Stats [default] ** 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-17T09:22:18.293 INFO:journalctl@ceph.mon.b.smithi157.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: ** Compaction Stats [default] ** 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-17T09:22:18.294 INFO:journalctl@ceph.mon.b.smithi157.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout: Block cache BinnedLRUCache@0x55cf6b49d350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: using public_addr v2:172.21.15.157:0/0 -> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-09-17T09:22:18.295 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: starting mon.b rank -1 at public addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] at bind addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(???) e0 preinit fsid ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).mds e1 new map 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout: e1 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout: legacy client fscid: -1 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout: 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout: No filesystems configured 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-17T09:22:18.296 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3998140238' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:18.297 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi148:172.21.15.148=a;smithi148:[v2:172.21.15.148:3301,v1:172.21.15.148:6790]=c;smithi157:172.21.15.157=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: Saving service mon spec with placement smithi148:172.21.15.148=a;smithi148:[v2:172.21.15.148:3301,v1:172.21.15.148:6790]=c;smithi157:172.21.15.157=b;count:3 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: Deploying daemon mon.b on smithi157 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/3722377706' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 ceph-mon[27402]: mon.b@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 bash[27366]: 58bf671c333fea30bd71c4f379b44fcffcf8d7f8b331ac1fa6311341f8186f5e 2024-09-17T09:22:18.298 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:18 smithi157 systemd[1]: Started Ceph mon.b for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:22:19.025 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T09:22:19.026 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T09:22:19.232 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:22:23.650 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:22:23.650 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.650 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: mon.a calling monitor election 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: mon.b calling monitor election 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: monmap e2: 2 mons at {a=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: fsmap 2024-09-17T09:22:23.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:22:23.652 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: mgrmap e13: y(active, since 38s) 2024-09-17T09:22:23.652 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: overall HEALTH_OK 2024-09-17T09:22:23.652 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:23.652 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:23.652 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:23.652 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:23.652 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:23.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:22:23.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: mon.a calling monitor election 2024-09-17T09:22:23.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: mon.b calling monitor election 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: monmap e2: 2 mons at {a=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: fsmap 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:22:23.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: mgrmap e13: y(active, since 38s) 2024-09-17T09:22:23.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: overall HEALTH_OK 2024-09-17T09:22:23.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:23.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:23.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:23.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:23.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:24.055 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:22:24.056 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"ff0bb856-74d5-11ef-bceb-c7b262605968","modified":"2024-09-17T09:22:18.244230Z","created":"2024-09-17T09:20:49.437650Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-17T09:22:24.056 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-09-17T09:22:24.582 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:24 smithi157 ceph-mon[27402]: Deploying daemon mon.c on smithi148 2024-09-17T09:22:24.583 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:24 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/2498747179' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:22:24.583 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:24 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:24.650 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:24 smithi148 ceph-mon[25650]: Deploying daemon mon.c on smithi148 2024-09-17T09:22:24.650 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:24 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/2498747179' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:22:24.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:24 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:25.609 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-17T09:22:25.610 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-17T09:22:25.637 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:25 smithi157 ceph-mon[27402]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:25.646 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:25 smithi148 ceph-mon[25650]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:25.808 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:22:26.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:26 smithi148 systemd[1]: Started Ceph mon.c for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:22:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:22:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: mon.a calling monitor election 2024-09-17T09:22:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: mon.b calling monitor election 2024-09-17T09:22:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: mon.c calling monitor election 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: monmap e3: 3 mons at {a=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: fsmap 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: mgrmap e13: y(active, since 46s) 2024-09-17T09:22:31.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: overall HEALTH_OK 2024-09-17T09:22:31.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:22:31.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: mon.a calling monitor election 2024-09-17T09:22:31.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: mon.b calling monitor election 2024-09-17T09:22:31.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: mon.c calling monitor election 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T09:22:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: monmap e3: 3 mons at {a=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:22:31.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: fsmap 2024-09-17T09:22:31.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:22:31.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: mgrmap e13: y(active, since 46s) 2024-09-17T09:22:31.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: overall HEALTH_OK 2024-09-17T09:22:31.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:22:31.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: mon.a calling monitor election 2024-09-17T09:22:31.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: mon.b calling monitor election 2024-09-17T09:22:31.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:22:31.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:31.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: mon.c calling monitor election 2024-09-17T09:22:31.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:31.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: monmap e3: 3 mons at {a=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: fsmap 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: osdmap e4: 0 total, 0 up, 0 in 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: mgrmap e13: y(active, since 46s) 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: overall HEALTH_OK 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.922 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.922 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:31.929 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-17T09:22:31.929 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"fsid":"ff0bb856-74d5-11ef-bceb-c7b262605968","modified":"2024-09-17T09:22:26.472931Z","created":"2024-09-17T09:20:49.437650Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3301","nonce":0},{"type":"v1","addr":"172.21.15.148:6790","nonce":0}]},"addr":"172.21.15.148:6790/0","public_addr":"172.21.15.148:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-17T09:22:31.929 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 3 2024-09-17T09:22:32.516 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-17T09:22:32.516 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-17T09:22:32.744 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:32.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:32.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:32.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:32.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-17T09:22:32.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-17T09:22:32.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/2595391313' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:22:32.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:32.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:22:32.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:32.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:32.867 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:32.867 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:32.867 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/2595391313' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:32.868 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/2595391313' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-17T09:22:32.869 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.870 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.870 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.870 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.870 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:32.870 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:32.870 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:22:32.870 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:32.870 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:22:33.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:33 smithi157 ceph-mon[27402]: Updating smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:33.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:33 smithi157 ceph-mon[27402]: Updating smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:33.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:33 smithi157 ceph-mon[27402]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:22:33.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:33 smithi157 ceph-mon[27402]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T09:22:33.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:33 smithi157 ceph-mon[27402]: Reconfiguring daemon mon.a on smithi148 2024-09-17T09:22:33.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:33 smithi157 ceph-mon[27402]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:33.906 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[25650]: Updating smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:33.906 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[25650]: Updating smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:33.906 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[25650]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:22:33.906 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[25650]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T09:22:33.906 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[25650]: Reconfiguring daemon mon.a on smithi148 2024-09-17T09:22:33.906 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[25650]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:33.906 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[32020]: Updating smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:33.907 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[32020]: Updating smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:22:33.907 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[32020]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:22:33.907 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[32020]: Reconfiguring mon.a (unknown last config time)... 2024-09-17T09:22:33.907 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[32020]: Reconfiguring daemon mon.a on smithi148 2024-09-17T09:22:33.907 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:33 smithi148 ceph-mon[32020]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:35.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:35 smithi157 ceph-mon[27402]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:35.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.081 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.082 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.082 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.082 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[25650]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:35.082 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.082 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.082 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.082 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.082 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.082 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.082 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.083 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[32020]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:35.083 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.083 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.083 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.083 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:35.671 INFO:teuthology.orchestra.run.smithi148.stdout:# minimal ceph.conf for ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:22:35.671 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-09-17T09:22:35.671 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:22:35.671 INFO:teuthology.orchestra.run.smithi148.stdout: mon_host = [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] 2024-09-17T09:22:36.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:36.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:36 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/1704117989' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:36.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:36.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:36 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/1704117989' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:36.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:36.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:36 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/1704117989' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:36.827 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-17T09:22:36.827 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:22:36.827 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T09:22:36.855 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:22:36.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:36.921 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:22:36.921 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2024-09-17T09:22:36.956 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:22:36.956 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:22:37.024 INFO:tasks.cephadm:Adding mgr.y on smithi148 2024-09-17T09:22:37.025 INFO:tasks.cephadm:Adding mgr.x on smithi157 2024-09-17T09:22:37.025 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi148=y;smithi157=x' 2024-09-17T09:22:37.261 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:22:37.684 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:37 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:37.684 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:37 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:37.685 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:37 smithi157 ceph-mon[27402]: Reconfiguring mon.c (monmap changed)... 2024-09-17T09:22:37.685 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:37 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:37.685 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:37 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:22:37.685 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:37 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:37.685 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:37 smithi157 ceph-mon[27402]: Reconfiguring daemon mon.c on smithi148 2024-09-17T09:22:37.685 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:37 smithi157 ceph-mon[27402]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:37.693 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:37.693 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:37.693 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[25650]: Reconfiguring mon.c (monmap changed)... 2024-09-17T09:22:37.694 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:37.694 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:22:37.694 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:37.694 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[25650]: Reconfiguring daemon mon.c on smithi148 2024-09-17T09:22:37.694 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[25650]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:37.694 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:37.694 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:37.694 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[32020]: Reconfiguring mon.c (monmap changed)... 2024-09-17T09:22:37.694 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:37.694 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:22:37.694 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:37.695 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[32020]: Reconfiguring daemon mon.c on smithi148 2024-09-17T09:22:37.695 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:37 smithi148 ceph-mon[32020]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:38.633 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.633 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.634 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.634 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.634 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.634 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.634 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.634 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.634 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.634 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.634 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.634 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.661 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.661 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.661 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.661 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.661 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.662 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:38.926 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mgr update... 2024-09-17T09:22:39.449 DEBUG:teuthology.orchestra.run.smithi157:mgr.x> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.x.service 2024-09-17T09:22:39.451 INFO:tasks.cephadm:Deploying OSDs... 2024-09-17T09:22:39.451 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:22:39.451 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:22:39.468 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T09:22:39.468 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/nvme4n1 2024-09-17T09:22:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/nvme4n1 2024-09-17T09:22:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:22:39.525 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-17T09:22:39.526 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:22:39.526 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:22:39.526 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-17 09:22:37.669675214 +0000 2024-09-17T09:22:39.526 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-17 09:19:50.861646334 +0000 2024-09-17T09:22:39.526 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-17 09:19:50.861646334 +0000 2024-09-17T09:22:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-17 09:19:50.857646429 +0000 2024-09-17T09:22:39.526 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-17T09:22:39.588 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[25650]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:39.588 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:39.588 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:39.588 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:39.588 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:39.588 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:22:39.589 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:39.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[32020]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:39.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:39.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:39.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:39.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:39.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:22:39.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:39.591 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-17T09:22:39.591 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-17T09:22:39.592 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000230784 s, 2.2 MB/s 2024-09-17T09:22:39.593 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-17T09:22:39.650 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/nvme3n1 2024-09-17T09:22:39.686 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:39 smithi157 ceph-mon[27402]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:39.686 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:39.686 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:39.686 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:39.686 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-17T09:22:39.686 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-17T09:22:39.686 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/nvme3n1 2024-09-17T09:22:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:22:39.705 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 1028 Links: 1 Device type: 103,6 2024-09-17T09:22:39.705 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:22:39.705 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:22:39.705 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-17 09:22:37.659675451 +0000 2024-09-17T09:22:39.705 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-17 09:19:50.705650050 +0000 2024-09-17T09:22:39.706 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-17 09:19:50.705650050 +0000 2024-09-17T09:22:39.706 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-17 09:19:50.701650145 +0000 2024-09-17T09:22:39.706 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-17T09:22:39.769 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-17T09:22:39.769 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-17T09:22:39.769 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000216796 s, 2.4 MB/s 2024-09-17T09:22:39.770 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-17T09:22:39.826 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/nvme2n1 2024-09-17T09:22:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/nvme2n1 2024-09-17T09:22:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:22:39.882 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 1016 Links: 1 Device type: 103,4 2024-09-17T09:22:39.882 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:22:39.882 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:22:39.882 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-17 09:22:37.650675665 +0000 2024-09-17T09:22:39.882 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-17 09:19:50.568653313 +0000 2024-09-17T09:22:39.882 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-17 09:19:50.568653313 +0000 2024-09-17T09:22:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-17 09:19:50.565653385 +0000 2024-09-17T09:22:39.882 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-17T09:22:39.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-17T09:22:39.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-17T09:22:39.945 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000215604 s, 2.4 MB/s 2024-09-17T09:22:39.946 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-17T09:22:40.002 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/nvme1n1 2024-09-17T09:22:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/nvme1n1 2024-09-17T09:22:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:22:40.057 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-17T09:22:40.057 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:22:40.058 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:22:40.058 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-17 09:22:37.639675926 +0000 2024-09-17T09:22:40.058 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-17 09:19:50.423656767 +0000 2024-09-17T09:22:40.058 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-17 09:19:50.423656767 +0000 2024-09-17T09:22:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-17 09:19:50.417656910 +0000 2024-09-17T09:22:40.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-17T09:22:40.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-17T09:22:40.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-17T09:22:40.120 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000335095 s, 1.5 MB/s 2024-09-17T09:22:40.121 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-17T09:22:40.178 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:22:40.178 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T09:22:40.196 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-17T09:22:40.197 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/nvme4n1 2024-09-17T09:22:40.253 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/nvme4n1 2024-09-17T09:22:40.253 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:22:40.253 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 1039 Links: 1 Device type: 103,8 2024-09-17T09:22:40.253 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:22:40.253 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:22:40.253 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-17 09:22:34.124167614 +0000 2024-09-17T09:22:40.253 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-17 09:19:52.465100454 +0000 2024-09-17T09:22:40.253 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-17 09:19:52.465100454 +0000 2024-09-17T09:22:40.253 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-17 09:19:52.462100509 +0000 2024-09-17T09:22:40.254 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-17T09:22:40.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-17T09:22:40.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-17T09:22:40.318 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000108039 s, 4.7 MB/s 2024-09-17T09:22:40.319 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-17T09:22:40.375 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/nvme3n1 2024-09-17T09:22:40.431 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/nvme3n1 2024-09-17T09:22:40.431 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:22:40.431 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-17T09:22:40.431 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:22:40.431 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:22:40.431 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-17 09:22:34.116167758 +0000 2024-09-17T09:22:40.431 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-17 09:19:52.319103109 +0000 2024-09-17T09:22:40.431 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-17 09:19:52.319103109 +0000 2024-09-17T09:22:40.431 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-17 09:19:52.316103163 +0000 2024-09-17T09:22:40.432 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-17T09:22:40.495 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-17T09:22:40.495 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-17T09:22:40.495 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.0001239 s, 4.1 MB/s 2024-09-17T09:22:40.496 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-17T09:22:40.552 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/nvme2n1 2024-09-17T09:22:40.607 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/nvme2n1 2024-09-17T09:22:40.607 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:22:40.607 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 1016 Links: 1 Device type: 103,4 2024-09-17T09:22:40.607 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:22:40.607 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:22:40.608 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-17 09:22:34.108167903 +0000 2024-09-17T09:22:40.608 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-17 09:19:52.191105436 +0000 2024-09-17T09:22:40.608 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-17 09:19:52.191105436 +0000 2024-09-17T09:22:40.608 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-17 09:19:52.187105509 +0000 2024-09-17T09:22:40.608 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-17T09:22:40.668 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:40 smithi157 ceph-mon[27402]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi148=y;smithi157=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:40.669 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:40 smithi157 ceph-mon[27402]: Saving service mgr spec with placement smithi148=y;smithi157=x;count:2 2024-09-17T09:22:40.669 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:40 smithi157 ceph-mon[27402]: Reconfiguring mon.b (monmap changed)... 2024-09-17T09:22:40.669 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:40 smithi157 ceph-mon[27402]: Reconfiguring daemon mon.b on smithi157 2024-09-17T09:22:40.670 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-17T09:22:40.671 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-17T09:22:40.671 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000109806 s, 4.7 MB/s 2024-09-17T09:22:40.672 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-17T09:22:40.729 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/nvme1n1 2024-09-17T09:22:40.785 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/nvme1n1 2024-09-17T09:22:40.786 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-17T09:22:40.786 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 1003 Links: 1 Device type: 103,2 2024-09-17T09:22:40.786 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-17T09:22:40.786 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-17T09:22:40.786 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-17 09:22:34.100168046 +0000 2024-09-17T09:22:40.786 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-17 09:19:52.038108218 +0000 2024-09-17T09:22:40.786 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-17 09:19:52.038108218 +0000 2024-09-17T09:22:40.786 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-17 09:19:52.035108272 +0000 2024-09-17T09:22:40.786 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-17T09:22:40.850 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-17T09:22:40.850 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-17T09:22:40.850 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000109749 s, 4.7 MB/s 2024-09-17T09:22:40.851 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-17T09:22:40.909 INFO:tasks.cephadm:Deploying osd.0 on smithi148 with /dev/nvme1n1... 2024-09-17T09:22:40.909 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-17T09:22:40.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:40 smithi148 ceph-mon[25650]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi148=y;smithi157=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:40.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:40 smithi148 ceph-mon[25650]: Saving service mgr spec with placement smithi148=y;smithi157=x;count:2 2024-09-17T09:22:40.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:40 smithi148 ceph-mon[25650]: Reconfiguring mon.b (monmap changed)... 2024-09-17T09:22:40.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:40 smithi148 ceph-mon[25650]: Reconfiguring daemon mon.b on smithi157 2024-09-17T09:22:40.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:40 smithi148 ceph-mon[32020]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi148=y;smithi157=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:40.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:40 smithi148 ceph-mon[32020]: Saving service mgr spec with placement smithi148=y;smithi157=x;count:2 2024-09-17T09:22:40.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:40 smithi148 ceph-mon[32020]: Reconfiguring mon.b (monmap changed)... 2024-09-17T09:22:40.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:40 smithi148 ceph-mon[32020]: Reconfiguring daemon mon.b on smithi157 2024-09-17T09:22:41.096 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:41.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:41.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:41.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:41.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:41.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:41.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:41.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:41.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:22:41.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T09:22:41.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:22:41.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:41.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:41 smithi157 ceph-mon[27402]: Deploying daemon mgr.x on smithi157 2024-09-17T09:22:41.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:41.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:41.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:41.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:41.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:41.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:41.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:41.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:22:41.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T09:22:41.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:22:41.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:41.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[25650]: Deploying daemon mgr.x on smithi157 2024-09-17T09:22:41.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:41.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:41.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:41.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:41.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:41.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:41.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:41.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:22:41.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-17T09:22:41.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:22:41.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:41.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:41 smithi148 ceph-mon[32020]: Deploying daemon mgr.x on smithi157 2024-09-17T09:22:42.361 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 systemd[1]: Starting Ceph mgr.x for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:22:42.643 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 podman[29583]: 2024-09-17 09:22:42.26220815 +0000 UTC m=+0.015497434 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:22:42.643 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 podman[29583]: 2024-09-17 09:22:42.364818157 +0000 UTC m=+0.118107424 container create 2b87dc5c3396b9e46d4aa55ec1326fe1323c668158dc5ea1d8d8a5391ecd5d53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-09-17T09:22:42.643 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 podman[29583]: 2024-09-17 09:22:42.575511303 +0000 UTC m=+0.328800582 container init 2b87dc5c3396b9e46d4aa55ec1326fe1323c668158dc5ea1d8d8a5391ecd5d53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-17T09:22:42.643 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 podman[29583]: 2024-09-17 09:22:42.578762736 +0000 UTC m=+0.332052005 container start 2b87dc5c3396b9e46d4aa55ec1326fe1323c668158dc5ea1d8d8a5391ecd5d53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-17T09:22:42.644 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 ceph-mgr[29602]: -- 172.21.15.157:0/1879767874 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56128c85ad00 con 0x56128c82f000 2024-09-17T09:22:42.644 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 bash[29583]: 2b87dc5c3396b9e46d4aa55ec1326fe1323c668158dc5ea1d8d8a5391ecd5d53 2024-09-17T09:22:42.917 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 systemd[1]: Started Ceph mgr.x for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:22:42.917 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:42.701+0000 7f2cbb6e0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:22:42.917 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:42.759+0000 7f2cbb6e0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:22:42.917 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:42.915+0000 7f2cbb6e0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:22:42.927 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/nvme1n1 2024-09-17T09:22:42.927 INFO:teuthology.orchestra.run.smithi148.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T09:22:42.932 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-17T09:22:42.947 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-09-17T09:22:42.948 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-09-17T09:22:42.948 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127812 s, 820 MB/s 2024-09-17T09:22:42.948 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-09-17T09:22:43.235 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:42 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:42.980+0000 7f2cbb6e0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:22:43.235 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:43.127+0000 7f2cbb6e0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:22:43.523 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:43.283+0000 7f2cbb6e0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:22:43.588 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi148:/dev/nvme1n1 2024-09-17T09:22:43.862 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:43.863 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:43.864 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.864 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.864 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.864 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.864 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:43.864 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:43.864 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:43.864 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.864 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:22:43.864 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:22:43.865 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:43 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:43.865 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:43.600+0000 7f2cbb6e0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:22:43.865 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:43.660+0000 7f2cbb6e0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:22:43.865 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:43.865 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.865 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.866 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.866 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.866 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:43.866 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:43.866 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:43.866 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:43.866 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-17T09:22:43.866 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-17T09:22:43.866 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:44.115 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:43.860+0000 7f2cbb6e0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:22:44.276 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:22:44.974 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:44 smithi148 ceph-mon[25650]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:22:44.974 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:44 smithi148 ceph-mon[25650]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T09:22:44.974 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:44 smithi148 ceph-mon[25650]: Reconfiguring daemon mgr.y on smithi148 2024-09-17T09:22:44.975 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:44.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:44 smithi148 ceph-mon[32020]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:22:44.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:44 smithi148 ceph-mon[32020]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T09:22:44.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:44 smithi148 ceph-mon[32020]: Reconfiguring daemon mgr.y on smithi148 2024-09-17T09:22:44.975 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:44 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:45.005 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:44 smithi157 ceph-mon[27402]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:22:45.006 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:44 smithi157 ceph-mon[27402]: Reconfiguring mgr.y (unknown last config time)... 2024-09-17T09:22:45.006 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:44 smithi157 ceph-mon[27402]: Reconfiguring daemon mgr.y on smithi148 2024-09-17T09:22:45.006 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:44 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:45.006 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:44 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:44.845+0000 7f2cbb6e0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:22:45.006 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:44 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:44.923+0000 7f2cbb6e0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:22:45.744 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:45.694+0000 7f2cbb6e0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:22:45.744 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:45 smithi157 ceph-mon[27402]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:45.744 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:45 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:45.744 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:45 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:45.744 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:45 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:45.744 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:45 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:45.744 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:45 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:45.744 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:45 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:45.755 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[25650]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:45.755 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:45.755 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:45.756 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:45.756 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:45.756 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:45.756 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:45.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[32020]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:45.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:45.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:45.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:45.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:45.756 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:45.757 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:45 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:46.023 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:45.816+0000 7f2cbb6e0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:22:46.023 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:45.960+0000 7f2cbb6e0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:22:46.320 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:46.027+0000 7f2cbb6e0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:22:46.320 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:46.186+0000 7f2cbb6e0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:22:46.320 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:46.257+0000 7f2cbb6e0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:22:46.321 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:46.317+0000 7f2cbb6e0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:22:46.603 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:46.472+0000 7f2cbb6e0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:22:46.603 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:46.536+0000 7f2cbb6e0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:22:46.603 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:46.600+0000 7f2cbb6e0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:22:46.969 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:22:46.970 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:46.970 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:46.970 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:46.970 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:46.970 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:46.970 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:46.970 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:46.970 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:46.970 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:22:46.970 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:22:46.970 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:22:46.971 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:47.023 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:46.682+0000 7f2cbb6e0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:22:47.023 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:46.740+0000 7f2cbb6e0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:22:47.023 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:46.799+0000 7f2cbb6e0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:22:47.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:47.834 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[25650]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:47.834 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[25650]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:47.834 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[25650]: Standby manager daemon x started 2024-09-17T09:22:47.834 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[25650]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:22:47.834 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[25650]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:22:47.834 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[25650]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:22:47.834 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[25650]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:22:47.834 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[32020]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:47.835 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[32020]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:47.835 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[32020]: Standby manager daemon x started 2024-09-17T09:22:47.835 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[32020]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:22:47.835 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[32020]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:22:47.835 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[32020]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:22:47.835 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:47 smithi148 ceph-mon[32020]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:22:48.023 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:22:47 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:22:47.574+0000 7f2cbb6e0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:22:48.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:47 smithi157 ceph-mon[27402]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:22:48.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:47 smithi157 ceph-mon[27402]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:48.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:47 smithi157 ceph-mon[27402]: Standby manager daemon x started 2024-09-17T09:22:48.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:47 smithi157 ceph-mon[27402]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:22:48.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:47 smithi157 ceph-mon[27402]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:22:48.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:47 smithi157 ceph-mon[27402]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:22:48.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:47 smithi157 ceph-mon[27402]: from='mgr.? 172.21.15.157:0/1758416865' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:22:49.445 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[25650]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-17T09:22:49.445 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:22:49.445 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[25650]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:49.445 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3582803337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fb0818-3f41-4cb5-915b-9833deef49d5"}]: dispatch 2024-09-17T09:22:49.445 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3582803337' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83fb0818-3f41-4cb5-915b-9833deef49d5"}]': finished 2024-09-17T09:22:49.446 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[25650]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T09:22:49.446 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:22:49.446 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[32020]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-17T09:22:49.446 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:22:49.446 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[32020]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:49.446 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3582803337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fb0818-3f41-4cb5-915b-9833deef49d5"}]: dispatch 2024-09-17T09:22:49.447 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3582803337' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83fb0818-3f41-4cb5-915b-9833deef49d5"}]': finished 2024-09-17T09:22:49.447 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[32020]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T09:22:49.447 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:49 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:22:49.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:49 smithi157 ceph-mon[27402]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-17T09:22:49.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:49 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:22:49.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:49 smithi157 ceph-mon[27402]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:49.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:49 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3582803337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fb0818-3f41-4cb5-915b-9833deef49d5"}]: dispatch 2024-09-17T09:22:49.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:49 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3582803337' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83fb0818-3f41-4cb5-915b-9833deef49d5"}]': finished 2024-09-17T09:22:49.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:49 smithi157 ceph-mon[27402]: osdmap e5: 1 total, 0 up, 1 in 2024-09-17T09:22:49.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:49 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:22:50.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:50 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3058936864' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:22:50.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:50 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3058936864' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:22:50.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:50 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3058936864' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:22:51.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:51 smithi157 ceph-mon[27402]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:51.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:51 smithi148 ceph-mon[25650]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:51.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:51 smithi148 ceph-mon[32020]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.883 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:53 smithi148 ceph-mon[25650]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:53.884 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:53 smithi148 ceph-mon[32020]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:54.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:53 smithi157 ceph-mon[27402]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:55.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:55 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:55.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:55 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:55.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:55 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:55.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:55 smithi148 ceph-mon[32020]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:55.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:55 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:55.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:55 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:55.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:55 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:55.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:55 smithi148 ceph-mon[25650]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:55.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:55 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:55.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:55 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:55.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:55 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:22:55.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:55 smithi157 ceph-mon[27402]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:57.885 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:57 smithi148 ceph-mon[32020]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:57.885 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:57 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T09:22:57.885 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:57 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:57.886 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:57 smithi148 ceph-mon[25650]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:57.886 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:57 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T09:22:57.886 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:57 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:58.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:57 smithi157 ceph-mon[27402]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:58.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:57 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-17T09:22:58.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:57 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:22:58.842 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:58 smithi148 ceph-mon[25650]: Deploying daemon osd.0 on smithi148 2024-09-17T09:22:58.842 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:58 smithi148 ceph-mon[32020]: Deploying daemon osd.0 on smithi148 2024-09-17T09:22:59.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:58 smithi157 ceph-mon[27402]: Deploying daemon osd.0 on smithi148 2024-09-17T09:22:59.906 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:22:59 smithi148 ceph-mon[25650]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:22:59.907 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:22:59 smithi148 ceph-mon[32020]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:00.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:22:59 smithi157 ceph-mon[27402]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:01 smithi148 ceph-mon[25650]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:01.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:01 smithi148 ceph-mon[32020]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:02.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:01 smithi157 ceph-mon[27402]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:03.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[25650]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:03.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:03.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:03.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:03.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:03.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:03.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:03.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[32020]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:03.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:03.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:03.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:03.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:03.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:03.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:03 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:03.844 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:03 smithi157 ceph-mon[27402]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:03.844 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:03 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:03.845 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:03 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:03.845 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:03 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:03.845 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:03 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:03.845 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:03 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:03.845 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:03 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.406 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 0 on host 'smithi148' 2024-09-17T09:23:05.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:05.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:05.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:05.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:05.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:05.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:05.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.891 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:05.891 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.891 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.891 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.892 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:05.892 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:05.892 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.892 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.892 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.892 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:05.892 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:05.892 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:05.988 DEBUG:teuthology.orchestra.run.smithi148:osd.0> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.0.service 2024-09-17T09:23:05.990 INFO:tasks.cephadm:Deploying osd.1 on smithi148 with /dev/nvme2n1... 2024-09-17T09:23:05.990 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-17T09:23:06.219 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:23:06.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:06 smithi148 ceph-mon[25650]: Adjusting osd_memory_target on smithi148 to 15040M 2024-09-17T09:23:06.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:06 smithi148 ceph-mon[25650]: from='osd.0 [v2:172.21.15.148:6802/751433583,v1:172.21.15.148:6803/751433583]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T09:23:06.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:06 smithi148 ceph-mon[25650]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T09:23:06.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:06 smithi148 ceph-mon[32020]: Adjusting osd_memory_target on smithi148 to 15040M 2024-09-17T09:23:06.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:06 smithi148 ceph-mon[32020]: from='osd.0 [v2:172.21.15.148:6802/751433583,v1:172.21.15.148:6803/751433583]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T09:23:06.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:06 smithi148 ceph-mon[32020]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T09:23:07.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:06 smithi157 ceph-mon[27402]: Adjusting osd_memory_target on smithi148 to 15040M 2024-09-17T09:23:07.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:06 smithi157 ceph-mon[27402]: from='osd.0 [v2:172.21.15.148:6802/751433583,v1:172.21.15.148:6803/751433583]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T09:23:07.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:06 smithi157 ceph-mon[27402]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-17T09:23:07.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[25650]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:07.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[25650]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T09:23:07.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[25650]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T09:23:07.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:07.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[25650]: from='osd.0 [v2:172.21.15.148:6802/751433583,v1:172.21.15.148:6803/751433583]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:07.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[25650]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:07.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[32020]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:07.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[32020]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T09:23:07.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[32020]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T09:23:07.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:07.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[32020]: from='osd.0 [v2:172.21.15.148:6802/751433583,v1:172.21.15.148:6803/751433583]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:07.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-mon[32020]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:07.920 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0[36240]: 2024-09-17T09:23:07.595+0000 7f0e7bca4640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T09:23:07.920 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:23:07 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0[36240]: 2024-09-17T09:23:07.597+0000 7f0e77acd640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T09:23:07.997 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/nvme2n1 2024-09-17T09:23:07.997 INFO:teuthology.orchestra.run.smithi148.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T09:23:08.000 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-17T09:23:08.016 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-09-17T09:23:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-09-17T09:23:08.016 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0139014 s, 754 MB/s 2024-09-17T09:23:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-09-17T09:23:08.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:07 smithi157 ceph-mon[27402]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:08.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:07 smithi157 ceph-mon[27402]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-17T09:23:08.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:07 smithi157 ceph-mon[27402]: osdmap e6: 1 total, 0 up, 1 in 2024-09-17T09:23:08.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:07 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:08.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:07 smithi157 ceph-mon[27402]: from='osd.0 [v2:172.21.15.148:6802/751433583,v1:172.21.15.148:6803/751433583]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:08.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:07 smithi157 ceph-mon[27402]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:08.627 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi148:/dev/nvme2n1 2024-09-17T09:23:08.651 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:08 smithi148 ceph-mon[32020]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:23:08.651 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:08 smithi148 ceph-mon[32020]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T09:23:08.651 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:08 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:08.652 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:08 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:08.652 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:08 smithi148 ceph-mon[25650]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:23:08.652 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:08 smithi148 ceph-mon[25650]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T09:23:08.652 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:08 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:08.652 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:08 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:08.822 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:23:09.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:08 smithi157 ceph-mon[27402]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:23:09.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:08 smithi157 ceph-mon[27402]: osdmap e7: 1 total, 0 up, 1 in 2024-09-17T09:23:09.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:08 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:09.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:08 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:09.880 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[25650]: purged_snaps scrub starts 2024-09-17T09:23:09.880 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[25650]: purged_snaps scrub ok 2024-09-17T09:23:09.880 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[25650]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:09.880 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:09.881 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[25650]: osd.0 [v2:172.21.15.148:6802/751433583,v1:172.21.15.148:6803/751433583] boot 2024-09-17T09:23:09.881 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[25650]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T09:23:09.881 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:09.881 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[32020]: purged_snaps scrub starts 2024-09-17T09:23:09.881 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[32020]: purged_snaps scrub ok 2024-09-17T09:23:09.881 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[32020]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:09.881 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:09.881 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[32020]: osd.0 [v2:172.21.15.148:6802/751433583,v1:172.21.15.148:6803/751433583] boot 2024-09-17T09:23:09.881 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[32020]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T09:23:09.881 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:09 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:10.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:09 smithi157 ceph-mon[27402]: purged_snaps scrub starts 2024-09-17T09:23:10.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:09 smithi157 ceph-mon[27402]: purged_snaps scrub ok 2024-09-17T09:23:10.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:09 smithi157 ceph-mon[27402]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-17T09:23:10.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:09 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:10.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:09 smithi157 ceph-mon[27402]: osd.0 [v2:172.21.15.148:6802/751433583,v1:172.21.15.148:6803/751433583] boot 2024-09-17T09:23:10.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:09 smithi157 ceph-mon[27402]: osdmap e8: 1 total, 1 up, 1 in 2024-09-17T09:23:10.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:09 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:23:10.871 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:10 smithi148 ceph-mon[25650]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T09:23:10.871 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:10.872 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:10.872 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:10.872 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:10 smithi148 ceph-mon[32020]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T09:23:10.872 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:10.872 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:10.872 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:11.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:10 smithi157 ceph-mon[27402]: osdmap e9: 1 total, 1 up, 1 in 2024-09-17T09:23:11.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:11.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:11.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:11.788 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:11 smithi148 ceph-mon[25650]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:11.789 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:11 smithi148 ceph-mon[25650]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:11.789 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:11 smithi148 ceph-mon[32020]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:11.789 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:11 smithi148 ceph-mon[32020]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:12.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:11 smithi157 ceph-mon[27402]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:12.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:11 smithi157 ceph-mon[27402]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:13.829 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:13 smithi148 ceph-mon[32020]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:13.829 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:13 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/1607623933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f13721c-e03d-4900-b493-158129624dcb"}]: dispatch 2024-09-17T09:23:13.829 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:13 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/1607623933' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f13721c-e03d-4900-b493-158129624dcb"}]': finished 2024-09-17T09:23:13.829 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:13 smithi148 ceph-mon[32020]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T09:23:13.829 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:13 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:13.830 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:13 smithi148 ceph-mon[25650]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:13.830 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:13 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/1607623933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f13721c-e03d-4900-b493-158129624dcb"}]: dispatch 2024-09-17T09:23:13.830 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:13 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/1607623933' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f13721c-e03d-4900-b493-158129624dcb"}]': finished 2024-09-17T09:23:13.830 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:13 smithi148 ceph-mon[25650]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T09:23:13.830 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:14.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:13 smithi157 ceph-mon[27402]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:14.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:13 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/1607623933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f13721c-e03d-4900-b493-158129624dcb"}]: dispatch 2024-09-17T09:23:14.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:13 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/1607623933' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f13721c-e03d-4900-b493-158129624dcb"}]': finished 2024-09-17T09:23:14.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:13 smithi157 ceph-mon[27402]: osdmap e10: 2 total, 1 up, 2 in 2024-09-17T09:23:14.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:13 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:15.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: Detected new or changed devices on smithi148 2024-09-17T09:23:15.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:15.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3447564856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: Detected new or changed devices on smithi148 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3447564856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:15.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:15.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: Detected new or changed devices on smithi148 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3447564856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:15.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:15.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:16.605 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:16 smithi148 ceph-mon[25650]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:16.606 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:16 smithi148 ceph-mon[32020]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:16.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:16 smithi157 ceph-mon[27402]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:18.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:18 smithi148 ceph-mon[25650]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:18.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:18 smithi148 ceph-mon[32020]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:18.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:18 smithi157 ceph-mon[27402]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:20.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:20 smithi148 ceph-mon[25650]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:20.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:20 smithi148 ceph-mon[32020]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:20.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:20 smithi157 ceph-mon[27402]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:21.542 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T09:23:21.543 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:21.543 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T09:23:21.543 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:21.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-17T09:23:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:22.607 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:22 smithi148 ceph-mon[25650]: Deploying daemon osd.1 on smithi148 2024-09-17T09:23:22.607 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:22 smithi148 ceph-mon[25650]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:22.608 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:22 smithi148 ceph-mon[32020]: Deploying daemon osd.1 on smithi148 2024-09-17T09:23:22.608 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:22 smithi148 ceph-mon[32020]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:22.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:22 smithi157 ceph-mon[27402]: Deploying daemon osd.1 on smithi148 2024-09-17T09:23:22.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:22 smithi157 ceph-mon[27402]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:24.465 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:24 smithi148 ceph-mon[25650]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:24.466 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:24 smithi148 ceph-mon[32020]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:24.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:24 smithi157 ceph-mon[27402]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:26.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:26 smithi148 ceph-mon[25650]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:26.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:26 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:26.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:26 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:26.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:26 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:26.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:26 smithi148 ceph-mon[32020]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:26.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:26 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:26.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:26 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:26.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:26 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:26.362 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:26 smithi157 ceph-mon[27402]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:26.363 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:26 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:26.363 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:26 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:26.363 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:26 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:27.328 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:27.328 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:27.328 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:27.328 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:27.329 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:27.329 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:27.329 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:27.329 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:27.329 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:27.329 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:27.329 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:27.329 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:27 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:27.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:27 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:27.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:27 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:27.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:27 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:27.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:27 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:27.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:27 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:27.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:27 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:28.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:28 smithi148 ceph-mon[25650]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:28.381 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:28 smithi148 ceph-mon[32020]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:28.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:28 smithi157 ceph-mon[27402]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:29.076 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 1 on host 'smithi148' 2024-09-17T09:23:29.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:29.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:29.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:29.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:29.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:29.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:29.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:29.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:29.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:29.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:29.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:29.746 DEBUG:teuthology.orchestra.run.smithi148:osd.1> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.1.service 2024-09-17T09:23:29.789 INFO:tasks.cephadm:Deploying osd.2 on smithi148 with /dev/nvme3n1... 2024-09-17T09:23:29.789 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-17T09:23:30.021 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:23:30.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:30 smithi148 ceph-mon[25650]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:30.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:30 smithi148 ceph-mon[25650]: Adjusting osd_memory_target on smithi148 to 7520M 2024-09-17T09:23:30.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:30 smithi148 ceph-mon[25650]: from='osd.1 [v2:172.21.15.148:6810/3073615842,v1:172.21.15.148:6811/3073615842]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:23:30.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:30 smithi148 ceph-mon[25650]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:23:30.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:30 smithi148 ceph-mon[32020]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:30.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:30 smithi148 ceph-mon[32020]: Adjusting osd_memory_target on smithi148 to 7520M 2024-09-17T09:23:30.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:30 smithi148 ceph-mon[32020]: from='osd.1 [v2:172.21.15.148:6810/3073615842,v1:172.21.15.148:6811/3073615842]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:23:30.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:30 smithi148 ceph-mon[32020]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:23:30.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:30 smithi157 ceph-mon[27402]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:30.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:30 smithi157 ceph-mon[27402]: Adjusting osd_memory_target on smithi148 to 7520M 2024-09-17T09:23:30.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:30 smithi157 ceph-mon[27402]: from='osd.1 [v2:172.21.15.148:6810/3073615842,v1:172.21.15.148:6811/3073615842]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:23:30.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:30 smithi157 ceph-mon[27402]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-17T09:23:31.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-mon[25650]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T09:23:31.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-mon[25650]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T09:23:31.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:31.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-mon[25650]: from='osd.1 [v2:172.21.15.148:6810/3073615842,v1:172.21.15.148:6811/3073615842]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:31.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-mon[25650]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:31.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-mon[32020]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T09:23:31.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-mon[32020]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T09:23:31.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:31.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-mon[32020]: from='osd.1 [v2:172.21.15.148:6810/3073615842,v1:172.21.15.148:6811/3073615842]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:31.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-mon[32020]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:31.418 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1[39707]: 2024-09-17T09:23:31.172+0000 7fd7c9ae4640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T09:23:31.418 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:23:31 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1[39707]: 2024-09-17T09:23:31.175+0000 7fd7c590d640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T09:23:31.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:31 smithi157 ceph-mon[27402]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-17T09:23:31.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:31 smithi157 ceph-mon[27402]: osdmap e11: 2 total, 1 up, 2 in 2024-09-17T09:23:31.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:31.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:31 smithi157 ceph-mon[27402]: from='osd.1 [v2:172.21.15.148:6810/3073615842,v1:172.21.15.148:6811/3073615842]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:31.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:31 smithi157 ceph-mon[27402]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:31.907 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/nvme3n1 2024-09-17T09:23:31.907 INFO:teuthology.orchestra.run.smithi148.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T09:23:31.911 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-17T09:23:31.928 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-09-17T09:23:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-09-17T09:23:31.928 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0152651 s, 687 MB/s 2024-09-17T09:23:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-09-17T09:23:32.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:32 smithi157 ceph-mon[27402]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:32.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:32 smithi157 ceph-mon[27402]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:23:32.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:32 smithi157 ceph-mon[27402]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T09:23:32.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:32.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:32.545 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:32 smithi148 ceph-mon[25650]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:32.545 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:32 smithi148 ceph-mon[25650]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:23:32.545 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:32 smithi148 ceph-mon[25650]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T09:23:32.545 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:32.545 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:32.545 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:32 smithi148 ceph-mon[32020]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:32.545 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:32 smithi148 ceph-mon[32020]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:23:32.545 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:32 smithi148 ceph-mon[32020]: osdmap e12: 2 total, 1 up, 2 in 2024-09-17T09:23:32.545 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:32.546 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:32.566 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi148:/dev/nvme3n1 2024-09-17T09:23:32.760 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:23:33.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:33 smithi148 ceph-mon[25650]: purged_snaps scrub starts 2024-09-17T09:23:33.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:33 smithi148 ceph-mon[25650]: purged_snaps scrub ok 2024-09-17T09:23:33.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:33 smithi148 ceph-mon[25650]: osd.1 [v2:172.21.15.148:6810/3073615842,v1:172.21.15.148:6811/3073615842] boot 2024-09-17T09:23:33.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:33 smithi148 ceph-mon[25650]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T09:23:33.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:33 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:33.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:33 smithi148 ceph-mon[32020]: purged_snaps scrub starts 2024-09-17T09:23:33.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:33 smithi148 ceph-mon[32020]: purged_snaps scrub ok 2024-09-17T09:23:33.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:33 smithi148 ceph-mon[32020]: osd.1 [v2:172.21.15.148:6810/3073615842,v1:172.21.15.148:6811/3073615842] boot 2024-09-17T09:23:33.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:33 smithi148 ceph-mon[32020]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T09:23:33.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:33 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:33.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:33 smithi157 ceph-mon[27402]: purged_snaps scrub starts 2024-09-17T09:23:33.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:33 smithi157 ceph-mon[27402]: purged_snaps scrub ok 2024-09-17T09:23:33.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:33 smithi157 ceph-mon[27402]: osd.1 [v2:172.21.15.148:6810/3073615842,v1:172.21.15.148:6811/3073615842] boot 2024-09-17T09:23:33.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:33 smithi157 ceph-mon[27402]: osdmap e13: 2 total, 2 up, 2 in 2024-09-17T09:23:33.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:33 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:23:34.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:34 smithi148 ceph-mon[25650]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:34.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:34 smithi148 ceph-mon[25650]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T09:23:34.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:34 smithi148 ceph-mon[32020]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:34.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:34 smithi148 ceph-mon[32020]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T09:23:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:34 smithi157 ceph-mon[27402]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-17T09:23:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:34 smithi157 ceph-mon[27402]: osdmap e14: 2 total, 2 up, 2 in 2024-09-17T09:23:35.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:35 smithi157 ceph-mon[27402]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:35.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:35.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:35.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:35.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:35.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:35.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:35.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:35.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:35.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:35.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:35.528 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[32020]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:35.528 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:35.528 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:35.528 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:35.528 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:35.528 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:35.528 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:35.528 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:35.528 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:35.528 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:35.528 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:35.529 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[25650]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:23:35.529 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:23:35.529 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:23:35.529 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:35.529 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:35.529 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:35.529 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:35.529 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:35.529 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:35.529 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:35.529 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:36.492 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:36 smithi148 ceph-mon[32020]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:36.492 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:36 smithi148 ceph-mon[32020]: Detected new or changed devices on smithi148 2024-09-17T09:23:36.492 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:36 smithi148 ceph-mon[32020]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:36.492 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:36 smithi148 ceph-mon[32020]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-17T09:23:36.493 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:36 smithi148 ceph-mon[25650]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:36.493 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:36 smithi148 ceph-mon[25650]: Detected new or changed devices on smithi148 2024-09-17T09:23:36.493 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:36 smithi148 ceph-mon[25650]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:36.493 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:36 smithi148 ceph-mon[25650]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-17T09:23:36.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:36 smithi157 ceph-mon[27402]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:36.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:36 smithi157 ceph-mon[27402]: Detected new or changed devices on smithi148 2024-09-17T09:23:36.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:36 smithi157 ceph-mon[27402]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:36.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:36 smithi157 ceph-mon[27402]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-17T09:23:38.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[25650]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:38.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:38.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:38.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:38.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:38.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:38.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:38.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:38.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[32020]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:38.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:38.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:38.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:38.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:38.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:38.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:38.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:38.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:38 smithi157 ceph-mon[27402]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:38.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:38.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:38.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:38.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:38.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:38.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:38.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:39.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:39 smithi148 ceph-mon[25650]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:39.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:39 smithi148 ceph-mon[25650]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-17T09:23:39.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:39 smithi148 ceph-mon[25650]: Cluster is now healthy 2024-09-17T09:23:39.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:39 smithi148 ceph-mon[32020]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:39.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:39 smithi148 ceph-mon[32020]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-17T09:23:39.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:39 smithi148 ceph-mon[32020]: Cluster is now healthy 2024-09-17T09:23:39.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:39 smithi157 ceph-mon[27402]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:39.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:39 smithi157 ceph-mon[27402]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-17T09:23:39.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:39 smithi157 ceph-mon[27402]: Cluster is now healthy 2024-09-17T09:23:40.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:40 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3003629380' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fefc97-1359-4473-bbd5-00ec879dbd3b"}]: dispatch 2024-09-17T09:23:40.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:40 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3003629380' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83fefc97-1359-4473-bbd5-00ec879dbd3b"}]': finished 2024-09-17T09:23:40.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:40 smithi157 ceph-mon[27402]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T09:23:40.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:40 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:40.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:40 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3003629380' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fefc97-1359-4473-bbd5-00ec879dbd3b"}]: dispatch 2024-09-17T09:23:40.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:40 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3003629380' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83fefc97-1359-4473-bbd5-00ec879dbd3b"}]': finished 2024-09-17T09:23:40.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:40 smithi148 ceph-mon[32020]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T09:23:40.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:40 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:40.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:40 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3003629380' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fefc97-1359-4473-bbd5-00ec879dbd3b"}]: dispatch 2024-09-17T09:23:40.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:40 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3003629380' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83fefc97-1359-4473-bbd5-00ec879dbd3b"}]': finished 2024-09-17T09:23:40.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:40 smithi148 ceph-mon[25650]: osdmap e15: 3 total, 2 up, 3 in 2024-09-17T09:23:40.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:40 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:41.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:41 smithi148 ceph-mon[32020]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:41.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:41 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3733213961' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:41.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:41 smithi148 ceph-mon[25650]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:41.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:41 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3733213961' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:41.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:41 smithi157 ceph-mon[27402]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:41.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:41 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3733213961' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:23:43.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:43 smithi157 ceph-mon[27402]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:43.505 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:43 smithi148 ceph-mon[25650]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:43.505 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:43 smithi148 ceph-mon[32020]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:45.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:45 smithi157 ceph-mon[27402]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:45.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:45 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:45.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:45 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:45.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:45 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:45.287 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:45 smithi148 ceph-mon[25650]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:45.287 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:45 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:45.288 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:45 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:45.288 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:45 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:45.288 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:45 smithi148 ceph-mon[32020]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:45.288 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:45 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:45.288 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:45 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:45.288 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:45 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:47.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:47 smithi157 ceph-mon[27402]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:47.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:47 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T09:23:47.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:47 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:47.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:47 smithi157 ceph-mon[27402]: Deploying daemon osd.2 on smithi148 2024-09-17T09:23:47.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:47 smithi148 ceph-mon[25650]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:47.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:47 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T09:23:47.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:47 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:47.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:47 smithi148 ceph-mon[25650]: Deploying daemon osd.2 on smithi148 2024-09-17T09:23:47.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:47 smithi148 ceph-mon[32020]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:47.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:47 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-17T09:23:47.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:47 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:47.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:47 smithi148 ceph-mon[32020]: Deploying daemon osd.2 on smithi148 2024-09-17T09:23:49.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:49 smithi148 ceph-mon[25650]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:49.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:49 smithi148 ceph-mon[32020]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:49.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:49 smithi157 ceph-mon[27402]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:51.368 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:51 smithi148 ceph-mon[25650]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:51.368 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:51 smithi148 ceph-mon[32020]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:51.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:51 smithi157 ceph-mon[27402]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:52.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:52.257 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:52.257 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:52.257 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:52.257 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:52.257 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:52.257 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:52.257 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:52 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:52.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:52 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:52.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:52 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:52.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:52 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:52.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:52 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:52.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:52 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:52.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:52 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:53.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:53 smithi148 ceph-mon[25650]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:53.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:53 smithi148 ceph-mon[32020]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:54.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:53 smithi157 ceph-mon[27402]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:54.279 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 2 on host 'smithi148' 2024-09-17T09:23:54.889 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:54.890 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:54.890 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:54.890 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:54.890 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:54.890 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:54.890 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:54.890 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:54.890 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:54.890 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:54.890 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:54.891 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:54.891 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:54.891 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:54.891 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:54.891 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:54.891 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:54.891 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:54.891 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:54.891 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:54.933 DEBUG:teuthology.orchestra.run.smithi148:osd.2> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.2.service 2024-09-17T09:23:54.936 INFO:tasks.cephadm:Deploying osd.3 on smithi148 with /dev/nvme4n1... 2024-09-17T09:23:54.936 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-17T09:23:55.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:55.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:55.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:55.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:55.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:55.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:23:55.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:55.026 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:55.026 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:55.026 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:55.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:55 smithi148 ceph-mon[25650]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:55.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:55 smithi148 ceph-mon[25650]: Adjusting osd_memory_target on smithi148 to 5013M 2024-09-17T09:23:55.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:55 smithi148 ceph-mon[25650]: from='osd.2 [v2:172.21.15.148:6818/1077590729,v1:172.21.15.148:6819/1077590729]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:23:55.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:55 smithi148 ceph-mon[25650]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:23:55.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:55 smithi148 ceph-mon[32020]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:55.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:55 smithi148 ceph-mon[32020]: Adjusting osd_memory_target on smithi148 to 5013M 2024-09-17T09:23:55.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:55 smithi148 ceph-mon[32020]: from='osd.2 [v2:172.21.15.148:6818/1077590729,v1:172.21.15.148:6819/1077590729]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:23:55.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:55 smithi148 ceph-mon[32020]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:23:55.951 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:23:56.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:55 smithi157 ceph-mon[27402]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:56.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:55 smithi157 ceph-mon[27402]: Adjusting osd_memory_target on smithi148 to 5013M 2024-09-17T09:23:56.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:55 smithi157 ceph-mon[27402]: from='osd.2 [v2:172.21.15.148:6818/1077590729,v1:172.21.15.148:6819/1077590729]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:23:56.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:55 smithi157 ceph-mon[27402]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-17T09:23:56.864 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-mon[25650]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T09:23:56.864 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-mon[25650]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T09:23:56.864 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:56.864 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-mon[25650]: from='osd.2 [v2:172.21.15.148:6818/1077590729,v1:172.21.15.148:6819/1077590729]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:56.864 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-mon[25650]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:56.864 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-mon[32020]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T09:23:56.865 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-mon[32020]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T09:23:56.865 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:56.865 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-mon[32020]: from='osd.2 [v2:172.21.15.148:6818/1077590729,v1:172.21.15.148:6819/1077590729]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:56.865 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-mon[32020]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:56.865 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2[43223]: 2024-09-17T09:23:56.648+0000 7f53c0f3c640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T09:23:56.865 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:23:56 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2[43223]: 2024-09-17T09:23:56.651+0000 7f53bcd65640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T09:23:57.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:56 smithi157 ceph-mon[27402]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-17T09:23:57.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:56 smithi157 ceph-mon[27402]: osdmap e16: 3 total, 2 up, 3 in 2024-09-17T09:23:57.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:56 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:57.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:56 smithi157 ceph-mon[27402]: from='osd.2 [v2:172.21.15.148:6818/1077590729,v1:172.21.15.148:6819/1077590729]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:57.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:56 smithi157 ceph-mon[27402]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:23:57.806 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:57 smithi148 ceph-mon[32020]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:57.806 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:57 smithi148 ceph-mon[32020]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:23:57.806 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:57 smithi148 ceph-mon[32020]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T09:23:57.806 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:57 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:57.806 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:57 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:57.807 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:57 smithi148 ceph-mon[25650]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:57.807 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:57 smithi148 ceph-mon[25650]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:23:57.807 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:57 smithi148 ceph-mon[25650]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T09:23:57.807 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:57 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:57.807 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:57 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:58.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:57 smithi157 ceph-mon[27402]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:23:58.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:57 smithi157 ceph-mon[27402]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:23:58.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:57 smithi157 ceph-mon[27402]: osdmap e17: 3 total, 2 up, 3 in 2024-09-17T09:23:58.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:57 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:58.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:57 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:58.197 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/nvme4n1 2024-09-17T09:23:58.197 INFO:teuthology.orchestra.run.smithi148.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T09:23:58.201 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-17T09:23:58.223 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-09-17T09:23:58.223 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-09-17T09:23:58.223 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0201746 s, 520 MB/s 2024-09-17T09:23:58.223 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-09-17T09:23:59.157 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: purged_snaps scrub starts 2024-09-17T09:23:59.157 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: purged_snaps scrub ok 2024-09-17T09:23:59.157 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:59.157 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: osd.2 [v2:172.21.15.148:6818/1077590729,v1:172.21.15.148:6819/1077590729] boot 2024-09-17T09:23:59.157 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T09:23:59.157 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:59.157 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.157 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.157 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:59.158 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:59.158 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:59.158 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.158 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.158 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: purged_snaps scrub starts 2024-09-17T09:23:59.158 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: purged_snaps scrub ok 2024-09-17T09:23:59.158 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:59.158 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: osd.2 [v2:172.21.15.148:6818/1077590729,v1:172.21.15.148:6819/1077590729] boot 2024-09-17T09:23:59.158 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T09:23:59.158 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:59.159 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.159 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.159 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:59.159 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:59.159 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:59.159 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.159 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:58 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: purged_snaps scrub starts 2024-09-17T09:23:59.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: purged_snaps scrub ok 2024-09-17T09:23:59.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:59.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: osd.2 [v2:172.21.15.148:6818/1077590729,v1:172.21.15.148:6819/1077590729] boot 2024-09-17T09:23:59.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: osdmap e18: 3 total, 3 up, 3 in 2024-09-17T09:23:59.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:23:59.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:23:59.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:23:59.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:23:59.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:58 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:23:59.461 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi148:/dev/nvme4n1 2024-09-17T09:23:59.995 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:24:00.118 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[25650]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:00.119 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[25650]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:00.119 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:00.119 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[25650]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-17T09:24:00.119 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:00.119 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[25650]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T09:24:00.119 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:00.119 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[32020]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:00.119 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[32020]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:00.119 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:00.119 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[32020]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-17T09:24:00.120 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:00.120 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[32020]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T09:24:00.120 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:23:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:00.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:59 smithi157 ceph-mon[27402]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:00.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:59 smithi157 ceph-mon[27402]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-17T09:24:00.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:00.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:59 smithi157 ceph-mon[27402]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-17T09:24:00.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:00.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:59 smithi157 ceph-mon[27402]: osdmap e19: 3 total, 3 up, 3 in 2024-09-17T09:24:00.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:23:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-17T09:24:01.075 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:01.075 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:00 smithi148 ceph-mon[25650]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T09:24:01.075 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:00 smithi148 ceph-mon[25650]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:01.075 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:01.075 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:00 smithi148 ceph-mon[32020]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T09:24:01.076 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:00 smithi148 ceph-mon[32020]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:01.076 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44686]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-17T09:24:01.076 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44686]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:01.076 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44686]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:01.076 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44686]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:01.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-17T09:24:01.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:00 smithi157 ceph-mon[27402]: osdmap e20: 3 total, 3 up, 3 in 2024-09-17T09:24:01.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:00 smithi157 ceph-mon[27402]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:01.356 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44743]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-17T09:24:01.356 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44743]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:01.356 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44743]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:01.357 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44743]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:01.667 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44767]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2024-09-17T09:24:01.667 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44767]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:01.668 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44767]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:01.668 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44767]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:02.128 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44914]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme2n1 2024-09-17T09:24:02.128 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44914]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:02.128 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44914]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:02.129 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:24:01 smithi148 sudo[44914]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:02.129 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T09:24:02.129 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.129 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.129 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.129 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:02.129 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.129 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:02.129 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:02.130 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.130 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:02.130 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:02.130 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:02.130 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T09:24:02.130 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.130 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.130 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:02.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:02.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:02.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:02.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:02.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:02.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: osdmap e21: 3 total, 3 up, 3 in 2024-09-17T09:24:02.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:02.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:02.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:02.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:02.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:02.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:02.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:02.389 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[44990]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme3n1 2024-09-17T09:24:02.389 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[44990]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:02.389 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[44990]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:02.389 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[44990]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:02.660 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[44998]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme3n1 2024-09-17T09:24:02.661 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[44998]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:02.661 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[44998]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:02.661 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[44998]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:02.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[45015]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T09:24:02.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[45015]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:02.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[45015]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:02.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[45015]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:03.196 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:02 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:03.196 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:02 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:03.197 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:02 smithi148 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:03.201 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[45020]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T09:24:03.201 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[45020]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:03.201 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[45020]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:03.201 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:02 smithi148 sudo[45020]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:03.201 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:02 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:03.201 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:02 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:03.201 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:02 smithi148 ceph-mon[25650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:03.203 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:02 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:03.204 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:02 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:03.204 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:02 smithi157 ceph-mon[27402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:03.459 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:03 smithi157 sudo[31108]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T09:24:03.460 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:03 smithi157 sudo[31108]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:03.460 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:03 smithi157 sudo[31108]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:03.460 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:03 smithi157 sudo[31108]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:03.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:03 smithi157 sudo[31111]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T09:24:03.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:03 smithi157 sudo[31111]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:03.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:03 smithi157 sudo[31111]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:03.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:03 smithi157 sudo[31111]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:03.785 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:03 smithi148 sudo[45070]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T09:24:03.785 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:03 smithi148 sudo[45070]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:03.785 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:03 smithi148 sudo[45070]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:03.786 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:03 smithi148 sudo[45070]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:04.130 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 sudo[45100]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T09:24:04.130 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 sudo[45100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 sudo[45100]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 sudo[45100]: pam_unix(sudo:session): session closed for user root 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: Detected new or changed devices on smithi148 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: Cluster is now healthy 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:04.131 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:04.132 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:04.132 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:04.132 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:04.132 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:04.132 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:04.132 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:04.132 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:04.132 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:04.132 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: Detected new or changed devices on smithi148 2024-09-17T09:24:04.132 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: Cluster is now healthy 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:04.133 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:04.134 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:04.134 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:04.134 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:04.134 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:04 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:04.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: Detected new or changed devices on smithi148 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: Cluster is now healthy 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:04.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:04.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:04.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:04.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:04.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:24:04.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:24:04.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:04 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:24:05.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:05 smithi157 ceph-mon[27402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:05.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:05 smithi157 ceph-mon[27402]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:05.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:05 smithi157 ceph-mon[27402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:05.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:05 smithi157 ceph-mon[27402]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T09:24:05.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:05 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/7638602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "129eb88b-a0a7-4f47-8f1f-8442bca041c2"}]: dispatch 2024-09-17T09:24:05.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:05 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/7638602' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "129eb88b-a0a7-4f47-8f1f-8442bca041c2"}]': finished 2024-09-17T09:24:05.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:05 smithi157 ceph-mon[27402]: osdmap e22: 4 total, 3 up, 4 in 2024-09-17T09:24:05.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:05.341 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:05.341 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[32020]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:05.341 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:05.341 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[32020]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T09:24:05.341 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/7638602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "129eb88b-a0a7-4f47-8f1f-8442bca041c2"}]: dispatch 2024-09-17T09:24:05.341 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/7638602' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "129eb88b-a0a7-4f47-8f1f-8442bca041c2"}]': finished 2024-09-17T09:24:05.342 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[32020]: osdmap e22: 4 total, 3 up, 4 in 2024-09-17T09:24:05.342 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:05.342 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[25650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T09:24:05.342 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[25650]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:05.342 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[25650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T09:24:05.342 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[25650]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-17T09:24:05.342 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/7638602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "129eb88b-a0a7-4f47-8f1f-8442bca041c2"}]: dispatch 2024-09-17T09:24:05.342 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/7638602' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "129eb88b-a0a7-4f47-8f1f-8442bca041c2"}]': finished 2024-09-17T09:24:05.342 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[25650]: osdmap e22: 4 total, 3 up, 4 in 2024-09-17T09:24:05.342 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:06.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:06 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:06.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:06 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:06.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:06 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:06.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:06 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/2033830054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:06.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:06.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:06.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:06.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:06 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/2033830054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:06.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:06 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:06.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:06 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:06.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:06 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:06.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:06 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/2033830054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:07.411 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:07 smithi157 ceph-mon[27402]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:07.575 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:07 smithi148 ceph-mon[32020]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:07.576 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:07 smithi148 ceph-mon[25650]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:09.638 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:09 smithi148 ceph-mon[25650]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:09.638 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:09 smithi148 ceph-mon[32020]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:09.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:09 smithi157 ceph-mon[27402]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:11.540 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:11 smithi148 ceph-mon[25650]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-17T09:24:11.540 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-17T09:24:11.540 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:11.540 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:11 smithi148 ceph-mon[32020]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-17T09:24:11.540 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-17T09:24:11.540 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:11.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:11 smithi157 ceph-mon[27402]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-17T09:24:11.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-17T09:24:11.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:12.510 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:12 smithi148 ceph-mon[25650]: Deploying daemon osd.3 on smithi148 2024-09-17T09:24:12.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:12 smithi148 ceph-mon[32020]: Deploying daemon osd.3 on smithi148 2024-09-17T09:24:12.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:12 smithi157 ceph-mon[27402]: Deploying daemon osd.3 on smithi148 2024-09-17T09:24:13.473 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:13 smithi148 ceph-mon[25650]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-17T09:24:13.474 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:13 smithi148 ceph-mon[32020]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-17T09:24:13.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:13 smithi157 ceph-mon[27402]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-17T09:24:15.576 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:15 smithi148 ceph-mon[25650]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-17T09:24:15.576 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:15 smithi148 ceph-mon[32020]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-17T09:24:15.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:15 smithi157 ceph-mon[27402]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-17T09:24:17.526 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[32020]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-09-17T09:24:17.527 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:17.527 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:17.527 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:17.527 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:17.527 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:17.527 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:17.527 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[25650]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-09-17T09:24:17.527 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:17.528 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:17.528 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:17.528 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:17.528 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:17.528 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:17 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:17.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:17 smithi157 ceph-mon[27402]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-09-17T09:24:17.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:17 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:17.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:17 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:17.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:17 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:17.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:17 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:17.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:17 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:17.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:17 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:19.416 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:19 smithi148 ceph-mon[25650]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-17T09:24:19.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:19 smithi148 ceph-mon[32020]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-17T09:24:19.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:19 smithi157 ceph-mon[27402]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-17T09:24:20.501 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:20 smithi148 ceph-mon[32020]: from='osd.3 [v2:172.21.15.148:6826/3632250658,v1:172.21.15.148:6827/3632250658]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T09:24:20.501 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:20 smithi148 ceph-mon[32020]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T09:24:20.501 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:20 smithi148 ceph-mon[25650]: from='osd.3 [v2:172.21.15.148:6826/3632250658,v1:172.21.15.148:6827/3632250658]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T09:24:20.501 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:20 smithi148 ceph-mon[25650]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T09:24:20.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:20 smithi157 ceph-mon[27402]: from='osd.3 [v2:172.21.15.148:6826/3632250658,v1:172.21.15.148:6827/3632250658]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T09:24:20.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:20 smithi157 ceph-mon[27402]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-17T09:24:21.268 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 3 on host 'smithi148' 2024-09-17T09:24:21.588 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-17T09:24:21.588 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T09:24:21.588 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T09:24:21.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:21.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='osd.3 [v2:172.21.15.148:6826/3632250658,v1:172.21.15.148:6827/3632250658]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:24:21.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:24:21.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:21.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.589 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.590 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.590 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.590 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:21.590 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:21.590 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.590 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-17T09:24:21.590 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T09:24:21.590 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T09:24:21.590 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:21.590 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='osd.3 [v2:172.21.15.148:6826/3632250658,v1:172.21.15.148:6827/3632250658]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:21.591 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:21 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-17T09:24:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-17T09:24:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: osdmap e23: 4 total, 3 up, 4 in 2024-09-17T09:24:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='osd.3 [v2:172.21.15.148:6826/3632250658,v1:172.21.15.148:6827/3632250658]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:24:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-17T09:24:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:21.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:21.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:21.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:21 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:21.906 DEBUG:teuthology.orchestra.run.smithi148:osd.3> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.3.service 2024-09-17T09:24:21.908 INFO:tasks.cephadm:Deploying osd.4 on smithi157 with /dev/nvme1n1... 2024-09-17T09:24:21.909 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-17T09:24:22.116 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:24:22.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: Adjusting osd_memory_target on smithi148 to 3760M 2024-09-17T09:24:22.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:24:22.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T09:24:22.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:22.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:22.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:22.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:22.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:22.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:22.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:22.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:22.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:22.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: Adjusting osd_memory_target on smithi148 to 3760M 2024-09-17T09:24:22.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:24:22.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T09:24:22.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:22.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:22.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:22.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:22.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:22.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:22.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:22.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:22.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:22.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: Adjusting osd_memory_target on smithi148 to 3760M 2024-09-17T09:24:22.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-17T09:24:22.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: osdmap e24: 4 total, 3 up, 4 in 2024-09-17T09:24:22.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:22.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:22.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:22.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:22.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:22.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:22.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:22.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:22.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.436 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: purged_snaps scrub starts 2024-09-17T09:24:23.436 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: purged_snaps scrub ok 2024-09-17T09:24:23.436 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:23.436 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: Detected new or changed devices on smithi148 2024-09-17T09:24:23.436 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:23.436 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: osd.3 [v2:172.21.15.148:6826/3632250658,v1:172.21.15.148:6827/3632250658] boot 2024-09-17T09:24:23.436 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: osdmap e25: 4 total, 4 up, 4 in 2024-09-17T09:24:23.436 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:23.436 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.437 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.437 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:23.437 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.437 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:23.437 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:23.437 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: purged_snaps scrub starts 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: purged_snaps scrub ok 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: Detected new or changed devices on smithi148 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: osd.3 [v2:172.21.15.148:6826/3632250658,v1:172.21.15.148:6827/3632250658] boot 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: osdmap e25: 4 total, 4 up, 4 in 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: purged_snaps scrub starts 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: purged_snaps scrub ok 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: Detected new or changed devices on smithi148 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: osd.3 [v2:172.21.15.148:6826/3632250658,v1:172.21.15.148:6827/3632250658] boot 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: osdmap e25: 4 total, 4 up, 4 in 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:23.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:23.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:23.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:23.857 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/nvme1n1 2024-09-17T09:24:23.857 INFO:teuthology.orchestra.run.smithi157.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T09:24:23.859 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-17T09:24:23.871 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-09-17T09:24:23.872 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-09-17T09:24:23.872 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.010354 s, 1.0 GB/s 2024-09-17T09:24:23.872 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-09-17T09:24:24.426 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:/dev/nvme1n1 2024-09-17T09:24:24.620 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:24:24.739 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:24 smithi157 ceph-mon[27402]: osdmap e26: 4 total, 4 up, 4 in 2024-09-17T09:24:24.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:24 smithi148 ceph-mon[25650]: osdmap e26: 4 total, 4 up, 4 in 2024-09-17T09:24:24.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:24 smithi148 ceph-mon[32020]: osdmap e26: 4 total, 4 up, 4 in 2024-09-17T09:24:25.695 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:25 smithi157 ceph-mon[27402]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:25.695 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:25 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:25.695 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:25 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:25.695 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:25 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:25.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:25 smithi148 ceph-mon[25650]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:25.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:25 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:25.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:25 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:25.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:25 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:25.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:25 smithi148 ceph-mon[32020]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:25.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:25 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:25.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:25 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:25.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:25 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:27.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:27 smithi157 ceph-mon[27402]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:27.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:27 smithi148 ceph-mon[25650]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:27.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:27 smithi148 ceph-mon[32020]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:28.511 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:28 smithi157 ceph-mon[27402]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:28.511 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:28.511 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:28.511 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:28.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:28 smithi148 ceph-mon[25650]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:28.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:28.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:28.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:28.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:28 smithi148 ceph-mon[32020]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:28.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:28.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:28.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:29.577 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:29 smithi157 ceph-mon[27402]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:29.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:29 smithi148 ceph-mon[25650]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:29.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:29 smithi148 ceph-mon[32020]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:30.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:30 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/3369267631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65795d19-b30b-4df4-a5df-2a02b7fdeddf"}]: dispatch 2024-09-17T09:24:30.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:30 smithi157 ceph-mon[27402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65795d19-b30b-4df4-a5df-2a02b7fdeddf"}]: dispatch 2024-09-17T09:24:30.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:30 smithi157 ceph-mon[27402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65795d19-b30b-4df4-a5df-2a02b7fdeddf"}]': finished 2024-09-17T09:24:30.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:30 smithi157 ceph-mon[27402]: osdmap e27: 5 total, 4 up, 5 in 2024-09-17T09:24:30.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:30 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:30.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:30 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/3369267631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65795d19-b30b-4df4-a5df-2a02b7fdeddf"}]: dispatch 2024-09-17T09:24:30.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:30 smithi148 ceph-mon[25650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65795d19-b30b-4df4-a5df-2a02b7fdeddf"}]: dispatch 2024-09-17T09:24:30.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:30 smithi148 ceph-mon[25650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65795d19-b30b-4df4-a5df-2a02b7fdeddf"}]': finished 2024-09-17T09:24:30.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:30 smithi148 ceph-mon[25650]: osdmap e27: 5 total, 4 up, 5 in 2024-09-17T09:24:30.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:30 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:30.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:30 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/3369267631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65795d19-b30b-4df4-a5df-2a02b7fdeddf"}]: dispatch 2024-09-17T09:24:30.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:30 smithi148 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65795d19-b30b-4df4-a5df-2a02b7fdeddf"}]: dispatch 2024-09-17T09:24:30.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:30 smithi148 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65795d19-b30b-4df4-a5df-2a02b7fdeddf"}]': finished 2024-09-17T09:24:30.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:30 smithi148 ceph-mon[32020]: osdmap e27: 5 total, 4 up, 5 in 2024-09-17T09:24:30.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:30 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:31.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:31 smithi157 ceph-mon[27402]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:31.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:31 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/2720396857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:31.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:31 smithi148 ceph-mon[25650]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:31.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:31 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/2720396857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:31.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:31 smithi148 ceph-mon[32020]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:31.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:31 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/2720396857' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:33.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:33 smithi157 ceph-mon[27402]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:33.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:33 smithi148 ceph-mon[25650]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:33.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:33 smithi148 ceph-mon[32020]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:35.516 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:35 smithi157 ceph-mon[27402]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:35.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:35 smithi148 ceph-mon[25650]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:35.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:35 smithi148 ceph-mon[32020]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:37.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:37 smithi157 ceph-mon[27402]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:37.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:37 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-17T09:24:37.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:37 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:37.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:37 smithi157 ceph-mon[27402]: Deploying daemon osd.4 on smithi157 2024-09-17T09:24:37.792 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:37 smithi148 ceph-mon[32020]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:37.792 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:37 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-17T09:24:37.792 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:37 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:37.793 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:37 smithi148 ceph-mon[32020]: Deploying daemon osd.4 on smithi157 2024-09-17T09:24:37.793 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:37 smithi148 ceph-mon[25650]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:37.793 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:37 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-17T09:24:37.793 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:37 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:37.793 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:37 smithi148 ceph-mon[25650]: Deploying daemon osd.4 on smithi157 2024-09-17T09:24:39.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:39 smithi157 ceph-mon[27402]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:39.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:39.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:39.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:39.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:39 smithi148 ceph-mon[25650]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:39.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:39.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:39.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:39.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:39 smithi148 ceph-mon[32020]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:39.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:39.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:39.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:41.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:41 smithi157 ceph-mon[27402]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:41.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:41 smithi148 ceph-mon[32020]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:41.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:41 smithi148 ceph-mon[25650]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:42.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:42.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:42.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:42.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:42.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:42.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:42.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:42.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:42.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:42.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:42.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:42.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:42 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:42.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:42 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:42.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:42 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:42.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:42 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:42.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:42 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:42.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:42 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:42.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:42 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:43.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:43 smithi157 ceph-mon[27402]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:43.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:43 smithi148 ceph-mon[32020]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:43.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:43 smithi148 ceph-mon[25650]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:44.184 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 4 on host 'smithi157' 2024-09-17T09:24:44.733 DEBUG:teuthology.orchestra.run.smithi157:osd.4> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.4.service 2024-09-17T09:24:44.736 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/nvme2n1... 2024-09-17T09:24:44.736 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-17T09:24:44.756 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:44 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:44.756 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:44 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:44.756 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:44 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:44.756 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:44 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:44.757 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:44 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:44.757 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:44 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:44.757 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:44 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:44.757 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:44 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:44.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:44.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:44.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:44.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:44 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:45.526 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:24:45.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:45 smithi157 ceph-mon[27402]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:45.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:45 smithi157 ceph-mon[27402]: Adjusting osd_memory_target on smithi157 to 16064M 2024-09-17T09:24:45.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:45 smithi157 ceph-mon[27402]: from='osd.4 [v2:172.21.15.157:6800/392952001,v1:172.21.15.157:6801/392952001]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T09:24:45.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:45 smithi157 ceph-mon[27402]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T09:24:45.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:45 smithi148 ceph-mon[32020]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:45.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:45 smithi148 ceph-mon[32020]: Adjusting osd_memory_target on smithi157 to 16064M 2024-09-17T09:24:45.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:45 smithi148 ceph-mon[32020]: from='osd.4 [v2:172.21.15.157:6800/392952001,v1:172.21.15.157:6801/392952001]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T09:24:45.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:45 smithi148 ceph-mon[32020]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T09:24:45.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:45 smithi148 ceph-mon[25650]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:45.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:45 smithi148 ceph-mon[25650]: Adjusting osd_memory_target on smithi157 to 16064M 2024-09-17T09:24:45.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:45 smithi148 ceph-mon[25650]: from='osd.4 [v2:172.21.15.157:6800/392952001,v1:172.21.15.157:6801/392952001]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T09:24:45.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:45 smithi148 ceph-mon[25650]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-17T09:24:46.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: osdmap e28: 5 total, 4 up, 5 in 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='osd.4 [v2:172.21.15.157:6800/392952001,v1:172.21.15.157:6801/392952001]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:46.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:46.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:46.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T09:24:46.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: osdmap e28: 5 total, 4 up, 5 in 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='osd.4 [v2:172.21.15.157:6800/392952001,v1:172.21.15.157:6801/392952001]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:46.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:46.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:46.919 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-17T09:24:46.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: osdmap e28: 5 total, 4 up, 5 in 2024-09-17T09:24:46.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:46.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='osd.4 [v2:172.21.15.157:6800/392952001,v1:172.21.15.157:6801/392952001]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:24:46.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:24:46.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:24:46.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:47.042 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:24:46.774+0000 7f1c29bbf640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T09:24:47.043 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:24:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:24:46.778+0000 7f1c219e0640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T09:24:47.279 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/nvme2n1 2024-09-17T09:24:47.279 INFO:teuthology.orchestra.run.smithi157.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T09:24:47.282 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-17T09:24:47.294 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-09-17T09:24:47.294 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-09-17T09:24:47.294 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0100768 s, 1.0 GB/s 2024-09-17T09:24:47.294 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-09-17T09:24:47.598 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:47 smithi157 ceph-mon[27402]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:47.598 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:47 smithi157 ceph-mon[27402]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:24:47.598 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:47 smithi157 ceph-mon[27402]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T09:24:47.598 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:47 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:47.598 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:47 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:47.598 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:47 smithi157 ceph-mon[27402]: from='osd.4 ' entity='osd.4' 2024-09-17T09:24:47.899 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[32020]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:47.899 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[32020]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:24:47.899 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[32020]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T09:24:47.899 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:47.899 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:47.899 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[32020]: from='osd.4 ' entity='osd.4' 2024-09-17T09:24:47.899 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[25650]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:47.899 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[25650]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:24:47.899 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[25650]: osdmap e29: 5 total, 4 up, 5 in 2024-09-17T09:24:47.899 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:47.899 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:47.900 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:47 smithi148 ceph-mon[25650]: from='osd.4 ' entity='osd.4' 2024-09-17T09:24:47.911 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:/dev/nvme2n1 2024-09-17T09:24:48.341 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:24:48.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:48 smithi157 ceph-mon[27402]: purged_snaps scrub starts 2024-09-17T09:24:48.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:48 smithi157 ceph-mon[27402]: purged_snaps scrub ok 2024-09-17T09:24:48.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:48 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:48.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:48 smithi157 ceph-mon[27402]: osd.4 [v2:172.21.15.157:6800/392952001,v1:172.21.15.157:6801/392952001] boot 2024-09-17T09:24:48.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:48 smithi157 ceph-mon[27402]: osdmap e30: 5 total, 5 up, 5 in 2024-09-17T09:24:48.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:48 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:48.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[25650]: purged_snaps scrub starts 2024-09-17T09:24:48.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[25650]: purged_snaps scrub ok 2024-09-17T09:24:48.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:48.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[25650]: osd.4 [v2:172.21.15.157:6800/392952001,v1:172.21.15.157:6801/392952001] boot 2024-09-17T09:24:48.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[25650]: osdmap e30: 5 total, 5 up, 5 in 2024-09-17T09:24:48.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:48.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[32020]: purged_snaps scrub starts 2024-09-17T09:24:48.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[32020]: purged_snaps scrub ok 2024-09-17T09:24:48.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:48.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[32020]: osd.4 [v2:172.21.15.157:6800/392952001,v1:172.21.15.157:6801/392952001] boot 2024-09-17T09:24:48.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[32020]: osdmap e30: 5 total, 5 up, 5 in 2024-09-17T09:24:48.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:48 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:24:49.736 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:49 smithi157 ceph-mon[27402]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:49.736 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:49 smithi157 ceph-mon[27402]: osdmap e31: 5 total, 5 up, 5 in 2024-09-17T09:24:49.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:49 smithi148 ceph-mon[25650]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:49.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:49 smithi148 ceph-mon[25650]: osdmap e31: 5 total, 5 up, 5 in 2024-09-17T09:24:49.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:49 smithi148 ceph-mon[32020]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-17T09:24:49.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:49 smithi148 ceph-mon[32020]: osdmap e31: 5 total, 5 up, 5 in 2024-09-17T09:24:51.119 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:50 smithi157 ceph-mon[27402]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:24:51.120 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:50 smithi157 ceph-mon[27402]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T09:24:51.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:50 smithi148 ceph-mon[25650]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:24:51.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:50 smithi148 ceph-mon[25650]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T09:24:51.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:50 smithi148 ceph-mon[32020]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:24:51.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:50 smithi148 ceph-mon[32020]: osdmap e32: 5 total, 5 up, 5 in 2024-09-17T09:24:51.956 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:51 smithi157 ceph-mon[27402]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T09:24:51.956 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:51 smithi157 ceph-mon[27402]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:51.956 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:51 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:51.956 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:51 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:51.956 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:51 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:51.956 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:51 smithi157 ceph-mon[27402]: osdmap e34: 5 total, 5 up, 5 in 2024-09-17T09:24:52.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[25650]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T09:24:52.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[25650]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:52.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:52.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:52.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:52.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[25650]: osdmap e34: 5 total, 5 up, 5 in 2024-09-17T09:24:52.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[32020]: osdmap e33: 5 total, 5 up, 5 in 2024-09-17T09:24:52.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[32020]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:24:52.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:24:52.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:24:52.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:24:52.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:51 smithi148 ceph-mon[32020]: osdmap e34: 5 total, 5 up, 5 in 2024-09-17T09:24:53.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:52 smithi157 ceph-mon[27402]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:24:53.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:52 smithi148 ceph-mon[25650]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:24:53.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:52 smithi148 ceph-mon[32020]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:24:54.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:53 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/1493540686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f62bba-d563-407d-8dc7-f0f0948b6eab"}]: dispatch 2024-09-17T09:24:54.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:53 smithi157 ceph-mon[27402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f62bba-d563-407d-8dc7-f0f0948b6eab"}]: dispatch 2024-09-17T09:24:54.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:53 smithi157 ceph-mon[27402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90f62bba-d563-407d-8dc7-f0f0948b6eab"}]': finished 2024-09-17T09:24:54.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:53 smithi157 ceph-mon[27402]: osdmap e35: 6 total, 5 up, 6 in 2024-09-17T09:24:54.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:53 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:24:54.073 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:53 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/1493540686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f62bba-d563-407d-8dc7-f0f0948b6eab"}]: dispatch 2024-09-17T09:24:54.073 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:53 smithi148 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f62bba-d563-407d-8dc7-f0f0948b6eab"}]: dispatch 2024-09-17T09:24:54.073 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:53 smithi148 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90f62bba-d563-407d-8dc7-f0f0948b6eab"}]': finished 2024-09-17T09:24:54.073 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:53 smithi148 ceph-mon[32020]: osdmap e35: 6 total, 5 up, 6 in 2024-09-17T09:24:54.073 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:53 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:24:54.074 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:53 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/1493540686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f62bba-d563-407d-8dc7-f0f0948b6eab"}]: dispatch 2024-09-17T09:24:54.074 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:53 smithi148 ceph-mon[25650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f62bba-d563-407d-8dc7-f0f0948b6eab"}]: dispatch 2024-09-17T09:24:54.074 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:53 smithi148 ceph-mon[25650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90f62bba-d563-407d-8dc7-f0f0948b6eab"}]': finished 2024-09-17T09:24:54.074 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:53 smithi148 ceph-mon[25650]: osdmap e35: 6 total, 5 up, 6 in 2024-09-17T09:24:54.074 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:53 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:24:55.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:55 smithi148 ceph-mon[25650]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-17T09:24:55.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:55 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/2160741525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:55.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:55 smithi148 ceph-mon[32020]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-17T09:24:55.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:55 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/2160741525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:56.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:55 smithi157 ceph-mon[27402]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-17T09:24:56.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:55 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/2160741525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:24:57.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:57 smithi157 ceph-mon[27402]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:24:57.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:57 smithi148 ceph-mon[25650]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:24:57.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:57 smithi148 ceph-mon[32020]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:24:59.509 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:59 smithi157 ceph-mon[27402]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-17T09:24:59.509 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:59.509 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:59.509 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:24:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:59.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:59 smithi148 ceph-mon[25650]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-17T09:24:59.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:59.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:59.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:24:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:59.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:59 smithi148 ceph-mon[32020]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-17T09:24:59.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:59.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:24:59.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:24:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:00.451 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-17T09:25:00.451 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:00.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-17T09:25:00.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:00.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-17T09:25:00.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:01.414 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:01 smithi157 ceph-mon[27402]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T09:25:01.414 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:01 smithi157 ceph-mon[27402]: Deploying daemon osd.5 on smithi157 2024-09-17T09:25:01.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:01 smithi148 ceph-mon[32020]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T09:25:01.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:01 smithi148 ceph-mon[32020]: Deploying daemon osd.5 on smithi157 2024-09-17T09:25:01.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:01 smithi148 ceph-mon[25650]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T09:25:01.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:01 smithi148 ceph-mon[25650]: Deploying daemon osd.5 on smithi157 2024-09-17T09:25:03.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:03 smithi157 ceph-mon[27402]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T09:25:03.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:03 smithi148 ceph-mon[25650]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T09:25:03.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:03 smithi148 ceph-mon[32020]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T09:25:05.593 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:05 smithi157 ceph-mon[27402]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:05.593 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:05.593 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:05.593 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:05.593 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:05.593 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:05.594 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:05 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:05.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[25650]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:05.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:05.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:05.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:05.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:05.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:05.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[32020]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:05.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:05 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.111 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:06 smithi157 ceph-mon[27402]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:07.111 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:06 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.111 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:06 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.111 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:06 smithi157 ceph-mon[27402]: Detected new or changed devices on smithi157 2024-09-17T09:25:07.111 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:06 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:07.111 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:06 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.112 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:06 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:07.112 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:06 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:07.112 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:06 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[32020]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:07.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[32020]: Detected new or changed devices on smithi157 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[25650]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[25650]: Detected new or changed devices on smithi157 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:07.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:07.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:07.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:06 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:08.413 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2024-09-17T09:25:08.918 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:08 smithi157 ceph-mon[27402]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:08.918 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:08 smithi157 ceph-mon[27402]: from='osd.5 [v2:172.21.15.157:6808/3770095390,v1:172.21.15.157:6809/3770095390]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T09:25:08.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:08 smithi157 ceph-mon[27402]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T09:25:08.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:08 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:08.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:08 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:08.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:08 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:08.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:08 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:08.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:08 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:08.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:08 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:09.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[25650]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:09.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[25650]: from='osd.5 [v2:172.21.15.157:6808/3770095390,v1:172.21.15.157:6809/3770095390]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T09:25:09.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[25650]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T09:25:09.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:09.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:09.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:09.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:09.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:09.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:09.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[32020]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:09.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[32020]: from='osd.5 [v2:172.21.15.157:6808/3770095390,v1:172.21.15.157:6809/3770095390]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T09:25:09.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[32020]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-17T09:25:09.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:09.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:09.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:09.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:09.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:09.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:08 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:09.921 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:09 smithi157 ceph-mon[27402]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T09:25:09.921 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:09 smithi157 ceph-mon[27402]: osdmap e36: 6 total, 5 up, 6 in 2024-09-17T09:25:09.921 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:09 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:09.921 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:09 smithi157 ceph-mon[27402]: from='osd.5 [v2:172.21.15.157:6808/3770095390,v1:172.21.15.157:6809/3770095390]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:09.922 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:09 smithi157 ceph-mon[27402]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:10.129 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.5.service 2024-09-17T09:25:10.131 INFO:tasks.cephadm:Deploying osd.6 on smithi157 with /dev/nvme3n1... 2024-09-17T09:25:10.132 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-17T09:25:10.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:09 smithi148 ceph-mon[25650]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T09:25:10.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:09 smithi148 ceph-mon[25650]: osdmap e36: 6 total, 5 up, 6 in 2024-09-17T09:25:10.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:09 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:10.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:09 smithi148 ceph-mon[25650]: from='osd.5 [v2:172.21.15.157:6808/3770095390,v1:172.21.15.157:6809/3770095390]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:10.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:09 smithi148 ceph-mon[25650]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:10.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:09 smithi148 ceph-mon[32020]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-17T09:25:10.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:09 smithi148 ceph-mon[32020]: osdmap e36: 6 total, 5 up, 6 in 2024-09-17T09:25:10.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:09 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:10.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:09 smithi148 ceph-mon[32020]: from='osd.5 [v2:172.21.15.157:6808/3770095390,v1:172.21.15.157:6809/3770095390]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:10.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:09 smithi148 ceph-mon[32020]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:10.615 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:25:11.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:11.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:25:11.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: osdmap e37: 6 total, 5 up, 6 in 2024-09-17T09:25:11.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:11.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:11.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: from='osd.5 ' entity='osd.5' 2024-09-17T09:25:11.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:11.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:11.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:11.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:11.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:11.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:11.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:10 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:11.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:11.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:25:11.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: osdmap e37: 6 total, 5 up, 6 in 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: from='osd.5 ' entity='osd.5' 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:11.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:25:11.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: osdmap e37: 6 total, 5 up, 6 in 2024-09-17T09:25:11.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:11.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:11.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: from='osd.5 ' entity='osd.5' 2024-09-17T09:25:11.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:11.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:11.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:11.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:11.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:11.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:11.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:10 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:12.020 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:11 smithi157 ceph-mon[27402]: purged_snaps scrub starts 2024-09-17T09:25:12.021 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:11 smithi157 ceph-mon[27402]: purged_snaps scrub ok 2024-09-17T09:25:12.021 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:12.021 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:11 smithi157 ceph-mon[27402]: osd.5 [v2:172.21.15.157:6808/3770095390,v1:172.21.15.157:6809/3770095390] boot 2024-09-17T09:25:12.021 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:11 smithi157 ceph-mon[27402]: osdmap e38: 6 total, 6 up, 6 in 2024-09-17T09:25:12.021 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:12.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[32020]: purged_snaps scrub starts 2024-09-17T09:25:12.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[32020]: purged_snaps scrub ok 2024-09-17T09:25:12.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:12.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[32020]: osd.5 [v2:172.21.15.157:6808/3770095390,v1:172.21.15.157:6809/3770095390] boot 2024-09-17T09:25:12.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[32020]: osdmap e38: 6 total, 6 up, 6 in 2024-09-17T09:25:12.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:12.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[25650]: purged_snaps scrub starts 2024-09-17T09:25:12.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[25650]: purged_snaps scrub ok 2024-09-17T09:25:12.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:12.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[25650]: osd.5 [v2:172.21.15.157:6808/3770095390,v1:172.21.15.157:6809/3770095390] boot 2024-09-17T09:25:12.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[25650]: osdmap e38: 6 total, 6 up, 6 in 2024-09-17T09:25:12.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:25:12.967 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/nvme3n1 2024-09-17T09:25:12.967 INFO:teuthology.orchestra.run.smithi157.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T09:25:12.968 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:12 smithi157 ceph-mon[27402]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:12.968 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:12 smithi157 ceph-mon[27402]: osdmap e39: 6 total, 6 up, 6 in 2024-09-17T09:25:12.970 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-17T09:25:12.982 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-09-17T09:25:12.982 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-09-17T09:25:12.982 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0104358 s, 1.0 GB/s 2024-09-17T09:25:12.982 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-09-17T09:25:13.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:12 smithi148 ceph-mon[25650]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:13.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:12 smithi148 ceph-mon[25650]: osdmap e39: 6 total, 6 up, 6 in 2024-09-17T09:25:13.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:12 smithi148 ceph-mon[32020]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-17T09:25:13.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:12 smithi148 ceph-mon[32020]: osdmap e39: 6 total, 6 up, 6 in 2024-09-17T09:25:13.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:/dev/nvme3n1 2024-09-17T09:25:13.810 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:25:13.930 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:13 smithi157 ceph-mon[27402]: osdmap e40: 6 total, 6 up, 6 in 2024-09-17T09:25:14.040 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:13 smithi148 ceph-mon[25650]: osdmap e40: 6 total, 6 up, 6 in 2024-09-17T09:25:14.041 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:13 smithi148 ceph-mon[32020]: osdmap e40: 6 total, 6 up, 6 in 2024-09-17T09:25:14.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:14 smithi148 ceph-mon[25650]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:14.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:14 smithi148 ceph-mon[25650]: osdmap e41: 6 total, 6 up, 6 in 2024-09-17T09:25:14.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:14 smithi148 ceph-mon[32020]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:14.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:14 smithi148 ceph-mon[32020]: osdmap e41: 6 total, 6 up, 6 in 2024-09-17T09:25:15.119 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:14 smithi157 ceph-mon[27402]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:15.119 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:14 smithi157 ceph-mon[27402]: osdmap e41: 6 total, 6 up, 6 in 2024-09-17T09:25:16.101 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:15 smithi148 ceph-mon[25650]: osdmap e42: 6 total, 6 up, 6 in 2024-09-17T09:25:16.101 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:15 smithi148 ceph-mon[25650]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:16.101 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:25:16.102 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:25:16.102 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:15 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:16.102 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:15 smithi148 ceph-mon[32020]: osdmap e42: 6 total, 6 up, 6 in 2024-09-17T09:25:16.102 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:15 smithi148 ceph-mon[32020]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:16.102 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:25:16.102 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:25:16.102 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:15 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:16.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:15 smithi157 ceph-mon[27402]: osdmap e42: 6 total, 6 up, 6 in 2024-09-17T09:25:16.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:15 smithi157 ceph-mon[27402]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:16.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:25:16.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:25:16.141 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:15 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:16.908 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:16 smithi157 ceph-mon[27402]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:17.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:16 smithi148 ceph-mon[25650]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:17.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:16 smithi148 ceph-mon[32020]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:18.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:17 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/2793834676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17521d9e-6ccd-4bd1-a6a6-04b2f193f262"}]: dispatch 2024-09-17T09:25:18.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:17 smithi157 ceph-mon[27402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17521d9e-6ccd-4bd1-a6a6-04b2f193f262"}]: dispatch 2024-09-17T09:25:18.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:17 smithi157 ceph-mon[27402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17521d9e-6ccd-4bd1-a6a6-04b2f193f262"}]': finished 2024-09-17T09:25:18.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:17 smithi157 ceph-mon[27402]: osdmap e43: 7 total, 6 up, 7 in 2024-09-17T09:25:18.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:17 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:18.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:17 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/2793834676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17521d9e-6ccd-4bd1-a6a6-04b2f193f262"}]: dispatch 2024-09-17T09:25:18.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:17 smithi148 ceph-mon[25650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17521d9e-6ccd-4bd1-a6a6-04b2f193f262"}]: dispatch 2024-09-17T09:25:18.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:17 smithi148 ceph-mon[25650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17521d9e-6ccd-4bd1-a6a6-04b2f193f262"}]': finished 2024-09-17T09:25:18.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:17 smithi148 ceph-mon[25650]: osdmap e43: 7 total, 6 up, 7 in 2024-09-17T09:25:18.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:17 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:18.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:17 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/2793834676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17521d9e-6ccd-4bd1-a6a6-04b2f193f262"}]: dispatch 2024-09-17T09:25:18.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:17 smithi148 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17521d9e-6ccd-4bd1-a6a6-04b2f193f262"}]: dispatch 2024-09-17T09:25:18.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:17 smithi148 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17521d9e-6ccd-4bd1-a6a6-04b2f193f262"}]': finished 2024-09-17T09:25:18.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:17 smithi148 ceph-mon[32020]: osdmap e43: 7 total, 6 up, 7 in 2024-09-17T09:25:18.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:17 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:19.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:19 smithi148 ceph-mon[25650]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:19.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:19 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:19.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:19 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:19.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:19 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:19.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:19 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/2974185789' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:25:19.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:19 smithi148 ceph-mon[32020]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:19.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:19 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:19.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:19 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:19.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:19 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:19.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:19 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/2974185789' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:25:19.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:19 smithi157 ceph-mon[27402]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:19.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:19 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:19.777 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:19 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:19.777 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:19 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:19.777 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:19 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/2974185789' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:25:21.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:21 smithi148 ceph-mon[25650]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:21.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:21 smithi148 ceph-mon[32020]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:21.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:21 smithi157 ceph-mon[27402]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:23.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:23 smithi157 ceph-mon[27402]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:23.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:23 smithi148 ceph-mon[25650]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:23.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:23 smithi148 ceph-mon[32020]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:24.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:24 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-17T09:25:24.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:24 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:24.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:24 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-17T09:25:24.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:24 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:24.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:24 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-17T09:25:24.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:24 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:25.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:25 smithi148 ceph-mon[25650]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:25.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:25 smithi148 ceph-mon[25650]: Deploying daemon osd.6 on smithi157 2024-09-17T09:25:25.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:25 smithi148 ceph-mon[32020]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:25.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:25 smithi148 ceph-mon[32020]: Deploying daemon osd.6 on smithi157 2024-09-17T09:25:25.687 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:25 smithi157 ceph-mon[27402]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:25.687 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:25 smithi157 ceph-mon[27402]: Deploying daemon osd.6 on smithi157 2024-09-17T09:25:27.569 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:27 smithi157 ceph-mon[27402]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:27.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:27 smithi148 ceph-mon[25650]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:27.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:27 smithi148 ceph-mon[32020]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:29.591 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:29 smithi157 ceph-mon[27402]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:29.592 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:29.592 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:29.592 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:29.592 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:29.592 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:29.592 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:29 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:29.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[25650]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:29.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:29.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:29.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:29.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:29.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:29.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:29.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[32020]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:29.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:29.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:29.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:29.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:29.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:29.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:29 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:31 smithi157 ceph-mon[27402]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:31.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:31 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[25650]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:31.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[32020]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:31.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:31.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:31.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:31.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:31.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:31 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:32.676 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:32 smithi157 ceph-mon[27402]: Detected new or changed devices on smithi157 2024-09-17T09:25:32.676 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:32 smithi157 ceph-mon[27402]: from='osd.6 [v2:172.21.15.157:6816/3080568127,v1:172.21.15.157:6817/3080568127]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T09:25:32.677 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:32 smithi157 ceph-mon[27402]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T09:25:32.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:32 smithi148 ceph-mon[25650]: Detected new or changed devices on smithi157 2024-09-17T09:25:32.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:32 smithi148 ceph-mon[25650]: from='osd.6 [v2:172.21.15.157:6816/3080568127,v1:172.21.15.157:6817/3080568127]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T09:25:32.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:32 smithi148 ceph-mon[25650]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T09:25:32.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:32 smithi148 ceph-mon[32020]: Detected new or changed devices on smithi157 2024-09-17T09:25:32.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:32 smithi148 ceph-mon[32020]: from='osd.6 [v2:172.21.15.157:6816/3080568127,v1:172.21.15.157:6817/3080568127]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T09:25:32.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:32 smithi148 ceph-mon[32020]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-17T09:25:33.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:33 smithi157 ceph-mon[27402]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:33.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:33 smithi157 ceph-mon[27402]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T09:25:33.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:33 smithi157 ceph-mon[27402]: osdmap e44: 7 total, 6 up, 7 in 2024-09-17T09:25:33.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:33 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:33.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:33 smithi157 ceph-mon[27402]: from='osd.6 [v2:172.21.15.157:6816/3080568127,v1:172.21.15.157:6817/3080568127]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:33.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:33 smithi157 ceph-mon[27402]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:33.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[25650]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:33.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[25650]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T09:25:33.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[25650]: osdmap e44: 7 total, 6 up, 7 in 2024-09-17T09:25:33.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:33.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[25650]: from='osd.6 [v2:172.21.15.157:6816/3080568127,v1:172.21.15.157:6817/3080568127]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:33.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[25650]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:33.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[32020]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:33.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[32020]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-17T09:25:33.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[32020]: osdmap e44: 7 total, 6 up, 7 in 2024-09-17T09:25:33.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:33.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[32020]: from='osd.6 [v2:172.21.15.157:6816/3080568127,v1:172.21.15.157:6817/3080568127]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:33.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:33 smithi148 ceph-mon[32020]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:25:34.522 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:25:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: osdmap e45: 7 total, 6 up, 7 in 2024-09-17T09:25:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: from='osd.6 ' entity='osd.6' 2024-09-17T09:25:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:34.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:34 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:34.557 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 6 on host 'smithi157' 2024-09-17T09:25:34.681 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:25:34.681 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: osdmap e45: 7 total, 6 up, 7 in 2024-09-17T09:25:34.681 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:34.681 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:34.681 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: from='osd.6 ' entity='osd.6' 2024-09-17T09:25:34.681 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:34.681 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:34.681 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:34.681 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:34.681 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:34.681 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: osdmap e45: 7 total, 6 up, 7 in 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: from='osd.6 ' entity='osd.6' 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:34.682 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:34.683 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:34 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.130 DEBUG:teuthology.orchestra.run.smithi157:osd.6> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.6.service 2024-09-17T09:25:35.136 INFO:tasks.cephadm:Deploying osd.7 on smithi157 with /dev/nvme4n1... 2024-09-17T09:25:35.136 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-17T09:25:35.368 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:25:35.699 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: purged_snaps scrub starts 2024-09-17T09:25:35.699 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: purged_snaps scrub ok 2024-09-17T09:25:35.699 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:35.699 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: Detected new or changed devices on smithi157 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: Adjusting osd_memory_target on smithi157 to 5354M 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:35.700 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.701 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: osd.6 [v2:172.21.15.157:6816/3080568127,v1:172.21.15.157:6817/3080568127] boot 2024-09-17T09:25:35.701 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: osdmap e46: 7 total, 7 up, 7 in 2024-09-17T09:25:35.701 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:35.701 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: purged_snaps scrub starts 2024-09-17T09:25:35.701 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: purged_snaps scrub ok 2024-09-17T09:25:35.701 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:35.701 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: Detected new or changed devices on smithi157 2024-09-17T09:25:35.701 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:35.701 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:35.701 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:35.701 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.702 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:35.702 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.702 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:35.702 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: Adjusting osd_memory_target on smithi157 to 5354M 2024-09-17T09:25:35.702 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.702 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:35.702 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:35.702 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.702 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: osd.6 [v2:172.21.15.157:6816/3080568127,v1:172.21.15.157:6817/3080568127] boot 2024-09-17T09:25:35.702 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: osdmap e46: 7 total, 7 up, 7 in 2024-09-17T09:25:35.702 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:35.712 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: purged_snaps scrub starts 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: purged_snaps scrub ok 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: Detected new or changed devices on smithi157 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: Adjusting osd_memory_target on smithi157 to 5354M 2024-09-17T09:25:35.713 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.714 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:35.714 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:35.714 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:35.714 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: osd.6 [v2:172.21.15.157:6816/3080568127,v1:172.21.15.157:6817/3080568127] boot 2024-09-17T09:25:35.714 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: osdmap e46: 7 total, 7 up, 7 in 2024-09-17T09:25:35.714 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:25:36.865 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:36 smithi148 ceph-mon[32020]: osdmap e47: 7 total, 7 up, 7 in 2024-09-17T09:25:36.865 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:36 smithi148 ceph-mon[25650]: osdmap e47: 7 total, 7 up, 7 in 2024-09-17T09:25:36.947 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:36 smithi157 ceph-mon[27402]: osdmap e47: 7 total, 7 up, 7 in 2024-09-17T09:25:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/nvme4n1 2024-09-17T09:25:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-17T09:25:37.166 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-17T09:25:37.178 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-09-17T09:25:37.178 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-09-17T09:25:37.179 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0101096 s, 1.0 GB/s 2024-09-17T09:25:37.179 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-09-17T09:25:37.777 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi157:/dev/nvme4n1 2024-09-17T09:25:37.983 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:25:38.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:37 smithi157 ceph-mon[27402]: pgmap v156: 1 pgs: 1 unknown; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:38.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:37 smithi157 ceph-mon[27402]: osdmap e48: 7 total, 7 up, 7 in 2024-09-17T09:25:38.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:37 smithi148 ceph-mon[25650]: pgmap v156: 1 pgs: 1 unknown; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:38.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:37 smithi148 ceph-mon[25650]: osdmap e48: 7 total, 7 up, 7 in 2024-09-17T09:25:38.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:37 smithi148 ceph-mon[32020]: pgmap v156: 1 pgs: 1 unknown; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:38.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:37 smithi148 ceph-mon[32020]: osdmap e48: 7 total, 7 up, 7 in 2024-09-17T09:25:38.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:38 smithi148 ceph-mon[25650]: osdmap e49: 7 total, 7 up, 7 in 2024-09-17T09:25:38.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:38.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:38.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:38 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:38.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:38 smithi148 ceph-mon[32020]: osdmap e49: 7 total, 7 up, 7 in 2024-09-17T09:25:38.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:38.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:38.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:38 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:39.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:38 smithi157 ceph-mon[27402]: osdmap e49: 7 total, 7 up, 7 in 2024-09-17T09:25:39.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:39.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:39.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:38 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:40.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:39 smithi157 ceph-mon[27402]: pgmap v159: 1 pgs: 1 unknown; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:40.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:39 smithi157 ceph-mon[27402]: osdmap e50: 7 total, 7 up, 7 in 2024-09-17T09:25:40.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:25:40.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:25:40.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:40.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:39 smithi148 ceph-mon[25650]: pgmap v159: 1 pgs: 1 unknown; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:40.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:39 smithi148 ceph-mon[25650]: osdmap e50: 7 total, 7 up, 7 in 2024-09-17T09:25:40.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:25:40.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:25:40.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:40.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:39 smithi148 ceph-mon[32020]: pgmap v159: 1 pgs: 1 unknown; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:40.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:39 smithi148 ceph-mon[32020]: osdmap e50: 7 total, 7 up, 7 in 2024-09-17T09:25:40.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-17T09:25:40.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-17T09:25:40.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:41.017 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:40 smithi157 ceph-mon[27402]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:41.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:40 smithi148 ceph-mon[25650]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:41.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:40 smithi148 ceph-mon[32020]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:25:42.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:41 smithi157 ceph-mon[27402]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:42.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:41 smithi148 ceph-mon[25650]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:42.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:41 smithi148 ceph-mon[32020]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:43.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:42 smithi157 ceph-mon[27402]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:25:43.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:42 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/2842722026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4109e289-2bc0-414e-8cf0-24acaf243f2a"}]: dispatch 2024-09-17T09:25:43.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:42 smithi157 ceph-mon[27402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4109e289-2bc0-414e-8cf0-24acaf243f2a"}]: dispatch 2024-09-17T09:25:43.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:42 smithi157 ceph-mon[27402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4109e289-2bc0-414e-8cf0-24acaf243f2a"}]': finished 2024-09-17T09:25:43.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:42 smithi157 ceph-mon[27402]: osdmap e51: 8 total, 7 up, 8 in 2024-09-17T09:25:43.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:42 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:25:43.069 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[32020]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:25:43.069 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/2842722026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4109e289-2bc0-414e-8cf0-24acaf243f2a"}]: dispatch 2024-09-17T09:25:43.069 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4109e289-2bc0-414e-8cf0-24acaf243f2a"}]: dispatch 2024-09-17T09:25:43.069 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4109e289-2bc0-414e-8cf0-24acaf243f2a"}]': finished 2024-09-17T09:25:43.069 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[32020]: osdmap e51: 8 total, 7 up, 8 in 2024-09-17T09:25:43.069 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:25:43.069 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[25650]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:25:43.069 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/2842722026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4109e289-2bc0-414e-8cf0-24acaf243f2a"}]: dispatch 2024-09-17T09:25:43.069 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[25650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4109e289-2bc0-414e-8cf0-24acaf243f2a"}]: dispatch 2024-09-17T09:25:43.069 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[25650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4109e289-2bc0-414e-8cf0-24acaf243f2a"}]': finished 2024-09-17T09:25:43.070 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[25650]: osdmap e51: 8 total, 7 up, 8 in 2024-09-17T09:25:43.070 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:42 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:25:44.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:43 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/3051155143' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:25:44.088 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:43 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/3051155143' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:25:44.089 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:43 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/3051155143' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-17T09:25:44.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:44 smithi148 ceph-mon[25650]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:25:44.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:44 smithi148 ceph-mon[32020]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:25:45.158 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:44 smithi157 ceph-mon[27402]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:25:47.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:46 smithi157 ceph-mon[27402]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T09:25:47.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:46 smithi148 ceph-mon[25650]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T09:25:47.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:46 smithi148 ceph-mon[32020]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-17T09:25:48.981 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:48 smithi157 ceph-mon[27402]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-17T09:25:48.982 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:48 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T09:25:48.982 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:48 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:48.982 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:48 smithi157 ceph-mon[27402]: Deploying daemon osd.7 on smithi157 2024-09-17T09:25:49.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:48 smithi148 ceph-mon[25650]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-17T09:25:49.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:48 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T09:25:49.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:48 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:49.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:48 smithi148 ceph-mon[25650]: Deploying daemon osd.7 on smithi157 2024-09-17T09:25:49.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:48 smithi148 ceph-mon[32020]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-17T09:25:49.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:48 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T09:25:49.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:48 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:49.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:48 smithi148 ceph-mon[32020]: Deploying daemon osd.7 on smithi157 2024-09-17T09:25:51.051 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:50 smithi157 ceph-mon[27402]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T09:25:51.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:50 smithi148 ceph-mon[25650]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T09:25:51.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:50 smithi148 ceph-mon[32020]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-17T09:25:52.954 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:52 smithi157 ceph-mon[27402]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:53.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:52 smithi148 ceph-mon[25650]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:53.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:52 smithi148 ceph-mon[32020]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:55.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:55.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:55.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: Reconfiguring osd.7 (unknown last config time)... 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[25650]: Reconfiguring daemon osd.7 on smithi157 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: Reconfiguring osd.7 (unknown last config time)... 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:55.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:54 smithi148 ceph-mon[32020]: Reconfiguring daemon osd.7 on smithi157 2024-09-17T09:25:55.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:55.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:55.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:55.225 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.225 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:55.225 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:55.225 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: Reconfiguring osd.7 (unknown last config time)... 2024-09-17T09:25:55.225 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-17T09:25:55.225 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:55.225 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:54 smithi157 ceph-mon[27402]: Reconfiguring daemon osd.7 on smithi157 2024-09-17T09:25:56.264 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:55 smithi157 ceph-mon[27402]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-17T09:25:56.264 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:55 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:56.264 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:55 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:56.380 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:55 smithi148 ceph-mon[32020]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-17T09:25:56.380 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:55 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:56.380 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:55 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:56.380 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:55 smithi148 ceph-mon[25650]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-17T09:25:56.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:55 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:56.381 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:55 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:57.202 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:56 smithi157 ceph-mon[27402]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:57.396 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:56 smithi148 ceph-mon[25650]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:57.396 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:56 smithi148 ceph-mon[32020]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:59.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:59.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: from='osd.7 [v2:172.21.15.157:6824/1756902340,v1:172.21.15.157:6825/1756902340]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T09:25:59.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T09:25:59.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:59.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:59.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:59.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:59.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: from='osd.7 [v2:172.21.15.157:6824/1756902340,v1:172.21.15.157:6825/1756902340]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:25:59 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.484 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:59.484 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: from='osd.7 [v2:172.21.15.157:6824/1756902340,v1:172.21.15.157:6825/1756902340]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T09:25:59.484 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-17T09:25:59.484 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.484 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.485 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:25:59.485 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:25:59.485 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:25:59.485 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:25:59.485 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.485 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.485 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:25:59.485 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:25:59 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.065 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 7 on host 'smithi157' 2024-09-17T09:26:00.345 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T09:26:00.345 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: osdmap e52: 8 total, 7 up, 8 in 2024-09-17T09:26:00.345 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:00.345 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='osd.7 [v2:172.21.15.157:6824/1756902340,v1:172.21.15.157:6825/1756902340]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:26:00.345 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:26:00.345 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-17T09:26:00.345 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: Cluster is now healthy 2024-09-17T09:26:00.345 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:00.346 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.346 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.346 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.346 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.346 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.346 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.346 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.346 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:00.346 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:00.346 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:00 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T09:26:00.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: osdmap e52: 8 total, 7 up, 8 in 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='osd.7 [v2:172.21.15.157:6824/1756902340,v1:172.21.15.157:6825/1756902340]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: Cluster is now healthy 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: osdmap e52: 8 total, 7 up, 8 in 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='osd.7 [v2:172.21.15.157:6824/1756902340,v1:172.21.15.157:6825/1756902340]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: Cluster is now healthy 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:00.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:00.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:00.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:00.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:00 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:00.658 DEBUG:teuthology.orchestra.run.smithi157:osd.7> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.7.service 2024-09-17T09:26:00.661 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-17T09:26:00.661 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-17T09:26:00.871 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:26:01.283 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[32020]: Adjusting osd_memory_target on smithi157 to 4016M 2024-09-17T09:26:01.283 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[32020]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:26:01.284 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[32020]: osdmap e53: 8 total, 7 up, 8 in 2024-09-17T09:26:01.284 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:01.284 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:01.284 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[32020]: from='osd.7 ' entity='osd.7' 2024-09-17T09:26:01.284 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[32020]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:26:01.285 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[32020]: osd.7 [v2:172.21.15.157:6824/1756902340,v1:172.21.15.157:6825/1756902340] boot 2024-09-17T09:26:01.285 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[32020]: osdmap e54: 8 total, 8 up, 8 in 2024-09-17T09:26:01.285 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:01.285 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[25650]: Adjusting osd_memory_target on smithi157 to 4016M 2024-09-17T09:26:01.286 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[25650]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:26:01.286 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[25650]: osdmap e53: 8 total, 7 up, 8 in 2024-09-17T09:26:01.286 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:01.286 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:01.286 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[25650]: from='osd.7 ' entity='osd.7' 2024-09-17T09:26:01.286 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[25650]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:26:01.287 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[25650]: osd.7 [v2:172.21.15.157:6824/1756902340,v1:172.21.15.157:6825/1756902340] boot 2024-09-17T09:26:01.287 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[25650]: osdmap e54: 8 total, 8 up, 8 in 2024-09-17T09:26:01.287 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:01 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:01.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:01 smithi157 ceph-mon[27402]: Adjusting osd_memory_target on smithi157 to 4016M 2024-09-17T09:26:01.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:01 smithi157 ceph-mon[27402]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-09-17T09:26:01.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:01 smithi157 ceph-mon[27402]: osdmap e53: 8 total, 7 up, 8 in 2024-09-17T09:26:01.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:01.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:01.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:01 smithi157 ceph-mon[27402]: from='osd.7 ' entity='osd.7' 2024-09-17T09:26:01.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:01 smithi157 ceph-mon[27402]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-17T09:26:01.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:01 smithi157 ceph-mon[27402]: osd.7 [v2:172.21.15.157:6824/1756902340,v1:172.21.15.157:6825/1756902340] boot 2024-09-17T09:26:01.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:01 smithi157 ceph-mon[27402]: osdmap e54: 8 total, 8 up, 8 in 2024-09-17T09:26:01.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:01 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:02.263 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:02 smithi148 ceph-mon[25650]: purged_snaps scrub starts 2024-09-17T09:26:02.263 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:02 smithi148 ceph-mon[25650]: purged_snaps scrub ok 2024-09-17T09:26:02.264 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:02 smithi148 ceph-mon[25650]: osdmap e55: 8 total, 8 up, 8 in 2024-09-17T09:26:02.264 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:02 smithi148 ceph-mon[32020]: purged_snaps scrub starts 2024-09-17T09:26:02.264 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:02 smithi148 ceph-mon[32020]: purged_snaps scrub ok 2024-09-17T09:26:02.264 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:02 smithi148 ceph-mon[32020]: osdmap e55: 8 total, 8 up, 8 in 2024-09-17T09:26:02.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:02 smithi157 ceph-mon[27402]: purged_snaps scrub starts 2024-09-17T09:26:02.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:02 smithi157 ceph-mon[27402]: purged_snaps scrub ok 2024-09-17T09:26:02.524 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:02 smithi157 ceph-mon[27402]: osdmap e55: 8 total, 8 up, 8 in 2024-09-17T09:26:02.626 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:26:03.347 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:03 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/4147507184' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T09:26:03.347 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:03 smithi148 ceph-mon[25650]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:03.347 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:03 smithi148 ceph-mon[25650]: osdmap e56: 8 total, 8 up, 8 in 2024-09-17T09:26:03.347 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:03 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/4147507184' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T09:26:03.347 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:03 smithi148 ceph-mon[32020]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:03.347 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:03 smithi148 ceph-mon[32020]: osdmap e56: 8 total, 8 up, 8 in 2024-09-17T09:26:03.368 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1726565161,"num_in_osds":8,"osd_in_since":1726565141,"num_remapped_pgs":1} 2024-09-17T09:26:03.368 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T09:26:03.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:03 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/4147507184' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-17T09:26:03.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:03 smithi157 ceph-mon[27402]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:03.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:03 smithi157 ceph-mon[27402]: osdmap e56: 8 total, 8 up, 8 in 2024-09-17T09:26:03.560 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:26:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:26:05.282 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":58,"fsid":"ff0bb856-74d5-11ef-bceb-c7b262605968","created":"2024-09-17T09:20:52.437233+0000","modified":"2024-09-17T09:26:05.179272+0000","last_up_change":"2024-09-17T09:26:01.156116+0000","last_in_change":"2024-09-17T09:25:41.855977+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T09:23:58.802065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83fb0818-3f41-4cb5-915b-9833deef49d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6803","nonce":751433583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6805","nonce":751433583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6809","nonce":751433583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6807","nonce":751433583}]},"public_addr":"172.21.15.148:6803/751433583","cluster_addr":"172.21.15.148:6805/751433583","heartbeat_back_addr":"172.21.15.148:6809/751433583","heartbeat_front_addr":"172.21.15.148:6807/751433583","state":["exists","up"]},{"osd":1,"uuid":"9f13721c-e03d-4900-b493-158129624dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6811","nonce":3073615842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6813","nonce":3073615842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6817","nonce":3073615842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6815","nonce":3073615842}]},"public_addr":"172.21.15.148:6811/3073615842","cluster_addr":"172.21.15.148:6813/3073615842","heartbeat_back_addr":"172.21.15.148:6817/3073615842","heartbeat_front_addr":"172.21.15.148:6815/3073615842","state":["exists","up"]},{"osd":2,"uuid":"83fefc97-1359-4473-bbd5-00ec879dbd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6819","nonce":1077590729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6821","nonce":1077590729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6825","nonce":1077590729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6823","nonce":1077590729}]},"public_addr":"172.21.15.148:6819/1077590729","cluster_addr":"172.21.15.148:6821/1077590729","heartbeat_back_addr":"172.21.15.148:6825/1077590729","heartbeat_front_addr":"172.21.15.148:6823/1077590729","state":["exists","up"]},{"osd":3,"uuid":"129eb88b-a0a7-4f47-8f1f-8442bca041c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6827","nonce":3632250658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6829","nonce":3632250658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6833","nonce":3632250658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6831","nonce":3632250658}]},"public_addr":"172.21.15.148:6827/3632250658","cluster_addr":"172.21.15.148:6829/3632250658","heartbeat_back_addr":"172.21.15.148:6833/3632250658","heartbeat_front_addr":"172.21.15.148:6831/3632250658","state":["exists","up"]},{"osd":4,"uuid":"65795d19-b30b-4df4-a5df-2a02b7fdeddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6801","nonce":392952001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6803","nonce":392952001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6807","nonce":392952001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6805","nonce":392952001}]},"public_addr":"172.21.15.157:6801/392952001","cluster_addr":"172.21.15.157:6803/392952001","heartbeat_back_addr":"172.21.15.157:6807/392952001","heartbeat_front_addr":"172.21.15.157:6805/392952001","state":["exists","up"]},{"osd":5,"uuid":"90f62bba-d563-407d-8dc7-f0f0948b6eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6809","nonce":3770095390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6811","nonce":3770095390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6815","nonce":3770095390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6813","nonce":3770095390}]},"public_addr":"172.21.15.157:6809/3770095390","cluster_addr":"172.21.15.157:6811/3770095390","heartbeat_back_addr":"172.21.15.157:6815/3770095390","heartbeat_front_addr":"172.21.15.157:6813/3770095390","state":["exists","up"]},{"osd":6,"uuid":"17521d9e-6ccd-4bd1-a6a6-04b2f193f262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6817","nonce":3080568127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6819","nonce":3080568127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6823","nonce":3080568127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6821","nonce":3080568127}]},"public_addr":"172.21.15.157:6817/3080568127","cluster_addr":"172.21.15.157:6819/3080568127","heartbeat_back_addr":"172.21.15.157:6823/3080568127","heartbeat_front_addr":"172.21.15.157:6821/3080568127","state":["exists","up"]},{"osd":7,"uuid":"4109e289-2bc0-414e-8cf0-24acaf243f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6825","nonce":1756902340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6827","nonce":1756902340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6831","nonce":1756902340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6829","nonce":1756902340}]},"public_addr":"172.21.15.157:6825/1756902340","cluster_addr":"172.21.15.157:6827/1756902340","heartbeat_back_addr":"172.21.15.157:6831/1756902340","heartbeat_front_addr":"172.21.15.157:6829/1756902340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:06.820629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:30.276799+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:55.775977+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:21.248005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:45.557104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:25:08.921480+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:25:33.373493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:25:59.484247+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:0/2957040738":"2024-09-18T09:21:44.497259+0000","172.21.15.148:0/3221867330":"2024-09-18T09:21:44.497259+0000","172.21.15.148:0/517099512":"2024-09-18T09:21:22.694776+0000","172.21.15.148:0/2186092680":"2024-09-18T09:21:22.694776+0000","172.21.15.148:0/2737221819":"2024-09-18T09:21:44.497259+0000","172.21.15.148:6801/2921533551":"2024-09-18T09:21:44.497259+0000","172.21.15.148:0/270732129":"2024-09-18T09:21:22.694776+0000","172.21.15.148:6801/3242443044":"2024-09-18T09:21:22.694776+0000","172.21.15.148:6800/2921533551":"2024-09-18T09:21:44.497259+0000","172.21.15.148:6800/3242443044":"2024-09-18T09:21:22.694776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T09:26:05.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:05 smithi157 ceph-mon[27402]: osdmap e57: 8 total, 8 up, 8 in 2024-09-17T09:26:05.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:05 smithi157 ceph-mon[27402]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:05.602 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:05 smithi148 ceph-mon[32020]: osdmap e57: 8 total, 8 up, 8 in 2024-09-17T09:26:05.602 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:05 smithi148 ceph-mon[32020]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:05.602 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:05 smithi148 ceph-mon[25650]: osdmap e57: 8 total, 8 up, 8 in 2024-09-17T09:26:05.602 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:05 smithi148 ceph-mon[25650]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:05.899 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T09:23:58.802065+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T09:26:05.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-17T09:26:06.093 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:26:06.481 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:06 smithi148 ceph-mon[25650]: osdmap e58: 8 total, 8 up, 8 in 2024-09-17T09:26:06.481 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:06 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/952506676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:26:06.481 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:06 smithi148 ceph-mon[32020]: osdmap e58: 8 total, 8 up, 8 in 2024-09-17T09:26:06.481 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:06 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/952506676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:26:06.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:06 smithi157 ceph-mon[27402]: osdmap e58: 8 total, 8 up, 8 in 2024-09-17T09:26:06.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:06 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/952506676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:26:07.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:07 smithi157 ceph-mon[27402]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-17T09:26:07.534 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:07 smithi148 ceph-mon[32020]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-17T09:26:07.534 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:07 smithi148 ceph-mon[25650]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-17T09:26:07.921 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-09-17T09:26:08.483 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:08 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/1342811730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T09:26:08.483 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:08 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/1342811730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T09:26:08.504 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi148 2024-09-17T09:26:08.504 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi148=foo.a' 2024-09-17T09:26:08.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:08 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/1342811730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-17T09:26:08.713 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:26:09.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:09 smithi157 ceph-mon[27402]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:26:09.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:09 smithi148 ceph-mon[25650]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:26:09.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:09 smithi148 ceph-mon[32020]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-17T09:26:10.496 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled rgw.foo.a update... 2024-09-17T09:26:11.087 DEBUG:teuthology.orchestra.run.smithi148:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-17T09:26:11.090 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi157 2024-09-17T09:26:11.090 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-17T09:26:11.346 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:26:11.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi148=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:11.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: Saving service rgw.foo.a spec with placement smithi148=foo.a;count:1 2024-09-17T09:26:11.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:11.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:11.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:11.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:11.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:11.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T09:26:11.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T09:26:11.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:11.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:11.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: Deploying daemon rgw.foo.a on smithi148 2024-09-17T09:26:11.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:11 smithi157 ceph-mon[27402]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-17T09:26:11.912 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi148=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:11.912 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: Saving service rgw.foo.a spec with placement smithi148=foo.a;count:1 2024-09-17T09:26:11.912 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:11.912 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:11.912 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:11.912 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:11.912 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:11.913 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T09:26:11.913 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T09:26:11.913 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:11.913 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:11.913 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: Deploying daemon rgw.foo.a on smithi148 2024-09-17T09:26:11.913 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[32020]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-17T09:26:11.913 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi148=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:11.913 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: Saving service rgw.foo.a spec with placement smithi148=foo.a;count:1 2024-09-17T09:26:11.913 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:11.913 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:11.913 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:11.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:11.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:11.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-17T09:26:11.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-17T09:26:11.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:11.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:11.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: Deploying daemon rgw.foo.a on smithi148 2024-09-17T09:26:11.914 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:11 smithi148 ceph-mon[25650]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-17T09:26:12.653 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:26:12 smithi148 systemd[1]: Starting Ceph rgw.foo.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:26:12.917 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:26:12 smithi148 podman[50532]: 2024-09-17 09:26:12.554587232 +0000 UTC m=+0.015558289 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:26:12.917 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:26:12 smithi148 podman[50532]: 2024-09-17 09:26:12.666919107 +0000 UTC m=+0.127890159 container create a7ed1a8a3f967f1e7793d1e5382b1f6eaa1804cb899b54cfc430158a6e4b9c17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-rgw-foo-a, ceph=True, GIT_CLEAN=True, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:26:12.917 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:26:12 smithi148 podman[50532]: 2024-09-17 09:26:12.894919881 +0000 UTC m=+0.355890934 container init a7ed1a8a3f967f1e7793d1e5382b1f6eaa1804cb899b54cfc430158a6e4b9c17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, ceph=True, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-17T09:26:12.917 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:26:12 smithi148 podman[50532]: 2024-09-17 09:26:12.897673704 +0000 UTC m=+0.358644752 container start a7ed1a8a3f967f1e7793d1e5382b1f6eaa1804cb899b54cfc430158a6e4b9c17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-rgw-foo-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-17T09:26:13.417 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:26:12 smithi148 bash[50532]: a7ed1a8a3f967f1e7793d1e5382b1f6eaa1804cb899b54cfc430158a6e4b9c17 2024-09-17T09:26:13.417 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:26:12 smithi148 systemd[1]: Started Ceph rgw.foo.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:26:13.999 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:13 smithi157 ceph-mon[27402]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-17T09:26:13.999 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:13 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:13.999 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:13 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:13.999 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:13 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:13.999 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:13 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.000 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:13 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.000 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:13 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:14.000 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:13 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:14.000 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:13 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:14.032 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[32020]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-17T09:26:14.032 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.033 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.033 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.033 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.033 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.033 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:14.033 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:14.033 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:14.033 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[25650]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-17T09:26:14.033 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.033 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.033 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.034 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.034 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.034 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:14.034 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:14.034 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:13 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:14.826 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: Saving service rgw.foo.a spec with placement smithi148=foo.a;count:1 2024-09-17T09:26:14.826 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: Checking dashboard <-> RGW credentials 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/927749195' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3685808707' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.827 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: Saving service rgw.foo.a spec with placement smithi148=foo.a;count:1 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: Checking dashboard <-> RGW credentials 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/927749195' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3685808707' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.828 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.829 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.829 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:14 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.914 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: Saving service rgw.foo.a spec with placement smithi148=foo.a;count:1 2024-09-17T09:26:14.915 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: Checking dashboard <-> RGW credentials 2024-09-17T09:26:14.915 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: osdmap e59: 8 total, 8 up, 8 in 2024-09-17T09:26:14.915 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/927749195' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.915 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3685808707' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.915 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.915 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-17T09:26:14.915 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.915 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.915 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.915 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.915 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.916 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:14.916 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:14 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:15.730 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'datapool' created 2024-09-17T09:26:15.918 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:15 smithi157 ceph-mon[27402]: pgmap v191: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-17T09:26:15.918 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:15 smithi157 ceph-mon[27402]: Detected new or changed devices on smithi157 2024-09-17T09:26:15.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:15 smithi157 ceph-mon[27402]: pgmap v192: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:15.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:15 smithi157 ceph-mon[27402]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T09:26:15.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:15 smithi157 ceph-mon[27402]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T09:26:15.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:15 smithi157 ceph-mon[27402]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T09:26:15.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:15 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/1581684372' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T09:26:15.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:15 smithi157 ceph-mon[27402]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T09:26:15.919 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:15 smithi157 ceph-mon[27402]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T09:26:16.030 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[25650]: pgmap v191: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-17T09:26:16.030 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[25650]: Detected new or changed devices on smithi157 2024-09-17T09:26:16.030 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[25650]: pgmap v192: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:16.030 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[25650]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T09:26:16.030 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[25650]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[25650]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/1581684372' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[25650]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[25650]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[32020]: pgmap v191: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[32020]: Detected new or changed devices on smithi157 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[32020]: pgmap v192: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[32020]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[32020]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[32020]: osdmap e60: 8 total, 8 up, 8 in 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/1581684372' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T09:26:16.031 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[32020]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T09:26:16.032 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:15 smithi148 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-17T09:26:16.502 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- rbd pool init datapool 2024-09-17T09:26:16.697 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:26:17.017 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:16 smithi157 ceph-mon[27402]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T09:26:17.017 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:16 smithi157 ceph-mon[27402]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T09:26:17.017 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:16 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3685808707' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:17.017 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:16 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/4086308826' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:17.017 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:16 smithi157 ceph-mon[27402]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:17.017 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:16 smithi157 ceph-mon[27402]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:17.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[25650]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T09:26:17.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[25650]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T09:26:17.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3685808707' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:17.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/4086308826' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:17.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[25650]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:17.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[25650]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:17.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-17T09:26:17.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[32020]: osdmap e61: 8 total, 8 up, 8 in 2024-09-17T09:26:17.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3685808707' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:17.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/4086308826' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:17.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[32020]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:17.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:16 smithi148 ceph-mon[32020]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-17T09:26:18.022 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:17 smithi157 ceph-mon[27402]: pgmap v195: 68 pgs: 8 creating+peering, 48 unknown, 12 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:18.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:17 smithi157 ceph-mon[27402]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:26:18.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:17 smithi157 ceph-mon[27402]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:26:18.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:17 smithi157 ceph-mon[27402]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T09:26:18.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:17 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:18.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:17 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:18.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:17 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:18.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:17 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:18.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[25650]: pgmap v195: 68 pgs: 8 creating+peering, 48 unknown, 12 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:18.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[25650]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:26:18.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[25650]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:26:18.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[25650]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T09:26:18.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:18.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:18.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:18.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:18.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[32020]: pgmap v195: 68 pgs: 8 creating+peering, 48 unknown, 12 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:26:18.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[32020]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:26:18.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[32020]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-17T09:26:18.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[32020]: osdmap e62: 8 total, 8 up, 8 in 2024-09-17T09:26:18.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:18.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:18.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:18.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:17 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:19.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:18 smithi157 ceph-mon[27402]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T09:26:19.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:18 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3685808707' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:18 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/4086308826' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:18 smithi157 ceph-mon[27402]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:18 smithi157 ceph-mon[27402]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:18 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/1356169761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T09:26:19.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:18 smithi157 ceph-mon[27402]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T09:26:19.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[32020]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T09:26:19.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3685808707' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/4086308826' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[32020]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[32020]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/1356169761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T09:26:19.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T09:26:19.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[25650]: osdmap e63: 8 total, 8 up, 8 in 2024-09-17T09:26:19.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3685808707' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/4086308826' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[25650]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[25650]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-17T09:26:19.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/1356169761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T09:26:19.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:18 smithi148 ceph-mon[25650]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-17T09:26:20.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:19 smithi157 ceph-mon[27402]: pgmap v198: 100 pgs: 16 creating+peering, 42 unknown, 42 active+clean; 578 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T09:26:20.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:19 smithi157 ceph-mon[27402]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:26:20.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:19 smithi157 ceph-mon[27402]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:26:20.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:19 smithi157 ceph-mon[27402]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T09:26:20.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:19 smithi157 ceph-mon[27402]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T09:26:20.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:19 smithi148 ceph-mon[32020]: pgmap v198: 100 pgs: 16 creating+peering, 42 unknown, 42 active+clean; 578 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T09:26:20.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:19 smithi148 ceph-mon[32020]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:26:20.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:19 smithi148 ceph-mon[32020]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:26:20.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:19 smithi148 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T09:26:20.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:19 smithi148 ceph-mon[32020]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T09:26:20.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:19 smithi148 ceph-mon[25650]: pgmap v198: 100 pgs: 16 creating+peering, 42 unknown, 42 active+clean; 578 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-17T09:26:20.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:19 smithi148 ceph-mon[25650]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:26:20.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:19 smithi148 ceph-mon[25650]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-17T09:26:20.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:19 smithi148 ceph-mon[25650]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-17T09:26:20.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:19 smithi148 ceph-mon[25650]: osdmap e64: 8 total, 8 up, 8 in 2024-09-17T09:26:21.077 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:20 smithi157 ceph-mon[27402]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T09:26:21.077 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:20 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/2348358891' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.077 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:20 smithi157 ceph-mon[27402]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.077 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:20 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/1223795140' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.077 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:20 smithi157 ceph-mon[27402]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:20 smithi148 ceph-mon[25650]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T09:26:21.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:20 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/2348358891' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:20 smithi148 ceph-mon[25650]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:20 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/1223795140' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:20 smithi148 ceph-mon[25650]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:20 smithi148 ceph-mon[32020]: osdmap e65: 8 total, 8 up, 8 in 2024-09-17T09:26:21.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:20 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/2348358891' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:20 smithi148 ceph-mon[32020]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:20 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/1223795140' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:20 smithi148 ceph-mon[32020]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-17T09:26:21.395 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.157 --placement '1;smithi157=iscsi.a' 2024-09-17T09:26:21.594 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:26:22.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:21 smithi157 ceph-mon[27402]: pgmap v201: 132 pgs: 23 creating+peering, 46 unknown, 63 active+clean; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T09:26:22.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:21 smithi157 ceph-mon[27402]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T09:26:22.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:21 smithi157 ceph-mon[27402]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:26:22.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:21 smithi157 ceph-mon[27402]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:26:22.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:21 smithi157 ceph-mon[27402]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T09:26:22.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:21 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/1223795140' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:22.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:21 smithi157 ceph-mon[27402]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:22.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:21 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/2348358891' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:22.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:21 smithi157 ceph-mon[27402]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:22.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[25650]: pgmap v201: 132 pgs: 23 creating+peering, 46 unknown, 63 active+clean; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T09:26:22.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[25650]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T09:26:22.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[25650]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:26:22.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[25650]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:26:22.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[25650]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T09:26:22.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/1223795140' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:22.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[25650]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:22.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/2348358891' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:22.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[25650]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:22.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[32020]: pgmap v201: 132 pgs: 23 creating+peering, 46 unknown, 63 active+clean; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-17T09:26:22.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[32020]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T09:26:22.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[32020]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:26:22.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[32020]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-17T09:26:22.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[32020]: osdmap e66: 8 total, 8 up, 8 in 2024-09-17T09:26:22.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/1223795140' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:22.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[32020]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:22.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/2348358891' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:22.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:21 smithi148 ceph-mon[32020]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-17T09:26:23.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:26:23.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:26:23.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T09:26:23.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:23.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:23.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:23.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: Checking dashboard <-> RGW credentials 2024-09-17T09:26:23.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:22 smithi157 ceph-mon[27402]: pgmap v204: 132 pgs: 3 creating+activating, 17 creating+peering, 22 unknown, 90 active+clean; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-17T09:26:23.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: Checking dashboard <-> RGW credentials 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[25650]: pgmap v204: 132 pgs: 3 creating+activating, 17 creating+peering, 22 unknown, 90 active+clean; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-17T09:26:23.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:26:23.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-17T09:26:23.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: osdmap e67: 8 total, 8 up, 8 in 2024-09-17T09:26:23.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:23.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:23.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:23.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: Checking dashboard <-> RGW credentials 2024-09-17T09:26:23.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:23.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:22 smithi148 ceph-mon[32020]: pgmap v204: 132 pgs: 3 creating+activating, 17 creating+peering, 22 unknown, 90 active+clean; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-17T09:26:23.451 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled iscsi.datapool update... 2024-09-17T09:26:24.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T09:26:24.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: Cluster is now healthy 2024-09-17T09:26:24.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.157", "placement": "1;smithi157=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:24.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: Saving service iscsi.datapool spec with placement smithi157=iscsi.a;count:1 2024-09-17T09:26:24.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:24.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:24.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:24.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:24.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:24.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-17T09:26:24.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T09:26:24.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:24.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[25650]: Deploying daemon iscsi.iscsi.a on smithi157 2024-09-17T09:26:24.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: Cluster is now healthy 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.157", "placement": "1;smithi157=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: Saving service iscsi.datapool spec with placement smithi157=iscsi.a;count:1 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:24.169 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:23 smithi148 ceph-mon[32020]: Deploying daemon iscsi.iscsi.a on smithi157 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: Cluster is now healthy 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.157", "placement": "1;smithi157=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: Saving service iscsi.datapool spec with placement smithi157=iscsi.a;count:1 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-17T09:26:24.186 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:24.187 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:23 smithi157 ceph-mon[27402]: Deploying daemon iscsi.iscsi.a on smithi157 2024-09-17T09:26:24.206 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-17T09:26:24.207 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:26:24.207 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-17T09:26:24.241 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:26:24.241 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-17T09:26:24.269 DEBUG:teuthology.orchestra.run.smithi157:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@iscsi.iscsi.a.service 2024-09-17T09:26:24.311 INFO:tasks.cephadm:Adding prometheus.a on smithi157 2024-09-17T09:26:24.312 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi157=a' 2024-09-17T09:26:25.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:24 smithi157 ceph-mon[27402]: pgmap v205: 132 pgs: 3 creating+activating, 5 unknown, 124 active+clean; 580 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 4.8 KiB/s wr, 97 op/s 2024-09-17T09:26:25.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:24 smithi148 ceph-mon[25650]: pgmap v205: 132 pgs: 3 creating+activating, 5 unknown, 124 active+clean; 580 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 4.8 KiB/s wr, 97 op/s 2024-09-17T09:26:25.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:24 smithi148 ceph-mon[32020]: pgmap v205: 132 pgs: 3 creating+activating, 5 unknown, 124 active+clean; 580 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 4.8 KiB/s wr, 97 op/s 2024-09-17T09:26:25.440 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:26:27.447 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:27 smithi157 ceph-mon[27402]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 5.7 KiB/s wr, 148 op/s 2024-09-17T09:26:27.448 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:27 smithi157 ceph-mon[27402]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T09:26:27.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:27 smithi148 ceph-mon[25650]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 5.7 KiB/s wr, 148 op/s 2024-09-17T09:26:27.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:27 smithi148 ceph-mon[25650]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T09:26:27.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:27 smithi148 ceph-mon[32020]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 5.7 KiB/s wr, 148 op/s 2024-09-17T09:26:27.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:27 smithi148 ceph-mon[32020]: osdmap e68: 8 total, 8 up, 8 in 2024-09-17T09:26:28.062 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled prometheus update... 2024-09-17T09:26:28.408 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug Processing osd blocklist entries for this node 2024-09-17T09:26:28.408 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug Reading the configuration object to update local LIO configuration 2024-09-17T09:26:28.408 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug Configuration does not have an entry for this host(smithi157.front.sepia.ceph.com) - nothing to define to LIO 2024-09-17T09:26:28.644 DEBUG:teuthology.orchestra.run.smithi157:prometheus.a> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-17T09:26:28.647 INFO:tasks.cephadm:Adding node-exporter.a on smithi148 2024-09-17T09:26:28.647 INFO:tasks.cephadm:Adding node-exporter.b on smithi157 2024-09-17T09:26:28.647 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '2;smithi148=a;smithi157=b' 2024-09-17T09:26:28.669 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-17T09:26:28.669 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: * Environment: production 2024-09-17T09:26:28.669 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T09:26:28.669 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: Use a production WSGI server instead. 2024-09-17T09:26:28.669 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: * Debug mode: off 2024-09-17T09:26:28.670 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug * Running on all addresses. 2024-09-17T09:26:28.670 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T09:26:28.670 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: * Running on all addresses. 2024-09-17T09:26:28.670 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-17T09:26:28.670 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug * Running on http://[fe80::ec4:7aff:fe88:7ea1]:5000/ (Press CTRL+C to quit) 2024-09-17T09:26:28.670 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: * Running on http://[fe80::ec4:7aff:fe88:7ea1]:5000/ (Press CTRL+C to quit) 2024-09-17T09:26:29.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T09:26:29.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:29.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: Saving service prometheus spec with placement smithi157=a;count:1 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: Adding iSCSI gateway http://:@172.21.15.157:5000 to Dashboard 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.275 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi157"}]: dispatch 2024-09-17T09:26:29.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi157"}]: dispatch 2024-09-17T09:26:29.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:29.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:29.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:29.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:26:29.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s rd, 5.1 KiB/s wr, 150 op/s 2024-09-17T09:26:29.276 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:28 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/3025117695' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T09:26:29.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: Saving service prometheus spec with placement smithi157=a;count:1 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.418 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: Adding iSCSI gateway http://:@172.21.15.157:5000 to Dashboard 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi157"}]: dispatch 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi157"}]: dispatch 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:26:29.419 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s rd, 5.1 KiB/s wr, 150 op/s 2024-09-17T09:26:29.420 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/3025117695' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T09:26:29.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-17T09:26:29.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:29.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: Saving service prometheus spec with placement smithi157=a;count:1 2024-09-17T09:26:29.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: Adding iSCSI gateway http://:@172.21.15.157:5000 to Dashboard 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi157"}]: dispatch 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi157"}]: dispatch 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s rd, 5.1 KiB/s wr, 150 op/s 2024-09-17T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:28 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/3025117695' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T09:26:30.161 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:26:30.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:30 smithi157 ceph-mon[27402]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T09:26:30.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:30 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:30.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:30 smithi148 ceph-mon[25650]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T09:26:30.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:30 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:30.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:30 smithi148 ceph-mon[32020]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-17T09:26:30.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:30 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:31.451 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:31 smithi157 ceph-mon[27402]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 KiB/s wr, 132 op/s 2024-09-17T09:26:31.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:31 smithi148 ceph-mon[25650]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 KiB/s wr, 132 op/s 2024-09-17T09:26:31.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:31 smithi148 ceph-mon[32020]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 KiB/s wr, 132 op/s 2024-09-17T09:26:32.645 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled node-exporter update... 2024-09-17T09:26:32.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:32 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:32.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:32 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:33.003 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:32 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:33.385 DEBUG:teuthology.orchestra.run.smithi148:node-exporter.a> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-17T09:26:33.388 DEBUG:teuthology.orchestra.run.smithi157:node-exporter.b> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-17T09:26:33.391 INFO:tasks.cephadm:Adding alertmanager.a on smithi148 2024-09-17T09:26:33.391 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi148=a' 2024-09-17T09:26:33.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[32020]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-09-17T09:26:33.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[32020]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi148=a;smithi157=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:33.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[32020]: Saving service node-exporter spec with placement smithi148=a;smithi157=b;count:2 2024-09-17T09:26:33.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:33.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:33.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:33.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:33.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:33.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[32020]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:26:33.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[25650]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-09-17T09:26:33.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[25650]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi148=a;smithi157=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:33.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[25650]: Saving service node-exporter spec with placement smithi148=a;smithi157=b;count:2 2024-09-17T09:26:33.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:33.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:33.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:33.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:33.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:33.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:33 smithi148 ceph-mon[25650]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:26:33.865 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:26:33.983 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:33 smithi157 ceph-mon[27402]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-09-17T09:26:33.983 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:33 smithi157 ceph-mon[27402]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi148=a;smithi157=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:33.983 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:33 smithi157 ceph-mon[27402]: Saving service node-exporter spec with placement smithi148=a;smithi157=b;count:2 2024-09-17T09:26:33.983 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:33 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:33.983 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:33 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:33.983 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:33 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:33.983 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:33 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:33.984 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:33 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:33.984 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:33 smithi157 ceph-mon[27402]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-17T09:26:35.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:35 smithi157 ceph-mon[27402]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 1.2 KiB/s wr, 60 op/s 2024-09-17T09:26:35.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:35 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:35.886 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:35 smithi148 ceph-mon[25650]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 1.2 KiB/s wr, 60 op/s 2024-09-17T09:26:35.886 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:35 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:35.887 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:35 smithi148 ceph-mon[32020]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 1.2 KiB/s wr, 60 op/s 2024-09-17T09:26:35.887 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:35 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:36.216 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled alertmanager update... 2024-09-17T09:26:36.813 DEBUG:teuthology.orchestra.run.smithi148:alertmanager.a> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-17T09:26:36.816 INFO:tasks.cephadm:Adding grafana.a on smithi157 2024-09-17T09:26:36.816 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi157=a' 2024-09-17T09:26:37.037 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:26:37.133 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:37.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:37.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:37.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: Deploying daemon prometheus.a on smithi157 2024-09-17T09:26:37.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi148=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:37.134 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: Saving service alertmanager spec with placement smithi148=a;count:1 2024-09-17T09:26:37.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 7.6 KiB/s rd, 102 B/s wr, 16 op/s 2024-09-17T09:26:37.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.135 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:36 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:37.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:37.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:37.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: Deploying daemon prometheus.a on smithi157 2024-09-17T09:26:37.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi148=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:37.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: Saving service alertmanager spec with placement smithi148=a;count:1 2024-09-17T09:26:37.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 7.6 KiB/s rd, 102 B/s wr, 16 op/s 2024-09-17T09:26:37.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:37.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:37.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:37.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: Deploying daemon prometheus.a on smithi157 2024-09-17T09:26:37.169 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi148=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:37.170 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: Saving service alertmanager spec with placement smithi148=a;count:1 2024-09-17T09:26:37.170 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.170 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 7.6 KiB/s rd, 102 B/s wr, 16 op/s 2024-09-17T09:26:37.170 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.170 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:37.170 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:36 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:38.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:38 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:26:39.431 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled grafana update... 2024-09-17T09:26:39.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:39 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:26:39.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:39 smithi157 ceph-mon[27402]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1012 B/s rd, 92 B/s wr, 1 op/s 2024-09-17T09:26:39.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:39 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:39.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:39 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:26:39.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:39 smithi148 ceph-mon[25650]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1012 B/s rd, 92 B/s wr, 1 op/s 2024-09-17T09:26:39.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:39 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:39.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:39 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:26:39.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:39 smithi148 ceph-mon[32020]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1012 B/s rd, 92 B/s wr, 1 op/s 2024-09-17T09:26:39.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:39 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:40.298 DEBUG:teuthology.orchestra.run.smithi157:grafana.a> sudo journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@grafana.a.service 2024-09-17T09:26:40.300 INFO:tasks.cephadm:Setting up client nodes... 2024-09-17T09:26:40.301 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T09:26:40.510 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:26:40.631 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:40 smithi148 ceph-mon[32020]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:40.631 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:40 smithi148 ceph-mon[32020]: Saving service grafana spec with placement smithi157=a;count:1 2024-09-17T09:26:40.631 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:40 smithi148 ceph-mon[25650]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:40.631 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:40 smithi148 ceph-mon[25650]: Saving service grafana spec with placement smithi157=a;count:1 2024-09-17T09:26:40.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:40 smithi157 ceph-mon[27402]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:26:40.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:40 smithi157 ceph-mon[27402]: Saving service grafana spec with placement smithi157=a;count:1 2024-09-17T09:26:41.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:41 smithi148 ceph-mon[25650]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:26:41.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:41 smithi148 ceph-mon[32020]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:26:41.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:41 smithi157 ceph-mon[27402]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:26:42.416 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-09-17T09:26:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBSS+lmshyQGBAAjvSPY8KdDM8VEZisYA6zuw== 2024-09-17T09:26:42.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:42 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3894862322' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:26:42.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:42 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3894862322' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:26:42.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:42 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3894862322' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:26:42.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:42 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3894862322' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:26:42.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:42 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3894862322' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:26:42.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:42 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3894862322' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:26:43.006 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:26:43.006 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-17T09:26:43.006 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T09:26:43.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-17T09:26:43.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:43 smithi157 ceph-mon[27402]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:26:43.810 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.b/config 2024-09-17T09:26:43.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:43 smithi148 ceph-mon[25650]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:26:43.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:43 smithi148 ceph-mon[32020]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:26:45.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:45 smithi157 ceph-mon[27402]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:26:45.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:45 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-17T09:26:45.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:45 smithi148 ceph-mon[25650]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:26:45.918 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:45 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-17T09:26:45.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:45 smithi148 ceph-mon[32020]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:26:45.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:45 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-17T09:26:46.756 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:46 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]': finished 2024-09-17T09:26:46.757 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:46 smithi157 ceph-mon[27402]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T09:26:46.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:46 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]': finished 2024-09-17T09:26:46.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:46 smithi148 ceph-mon[25650]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T09:26:46.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:46 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]': finished 2024-09-17T09:26:46.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:46 smithi148 ceph-mon[32020]: osdmap e69: 8 total, 8 up, 8 in 2024-09-17T09:26:47.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:47 smithi157 ceph-mon[27402]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T09:26:47.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:47 smithi157 ceph-mon[27402]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T09:26:47.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:47 smithi148 ceph-mon[25650]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T09:26:47.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:47 smithi148 ceph-mon[25650]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T09:26:47.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:47 smithi148 ceph-mon[32020]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-17T09:26:47.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:47 smithi148 ceph-mon[32020]: osdmap e70: 8 total, 8 up, 8 in 2024-09-17T09:26:48.518 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:48 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:26:49.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:49 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:26:49.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:49 smithi157 ceph-mon[27402]: pgmap v220: 132 pgs: 1 activating, 131 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:26:49.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:49 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:26:49.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:49 smithi148 ceph-mon[25650]: pgmap v220: 132 pgs: 1 activating, 131 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:26:49.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:49 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:26:49.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:49 smithi148 ceph-mon[32020]: pgmap v220: 132 pgs: 1 activating, 131 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:26:50.506 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2024-09-17T09:26:50.506 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBZS+lmlK31NRAAyZHbjX2DGq/EhmKqbxyCYg== 2024-09-17T09:26:50.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:50 smithi157 ceph-mon[27402]: from='client.? 172.21.15.157:0/2652337288' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:26:50.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:50 smithi157 ceph-mon[27402]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:26:50.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:50 smithi157 ceph-mon[27402]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:26:50.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:50 smithi148 ceph-mon[25650]: from='client.? 172.21.15.157:0/2652337288' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:26:50.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:50 smithi148 ceph-mon[25650]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:26:50.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:50 smithi148 ceph-mon[25650]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:26:50.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:50 smithi148 ceph-mon[32020]: from='client.? 172.21.15.157:0/2652337288' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:26:50.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:50 smithi148 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-17T09:26:50.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:50 smithi148 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-17T09:26:51.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:51 smithi157 ceph-mon[27402]: pgmap v221: 132 pgs: 1 activating, 131 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:26:51.884 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:26:51.885 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-17T09:26:51.885 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T09:26:51.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:51 smithi148 ceph-mon[25650]: pgmap v221: 132 pgs: 1 activating, 131 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:26:51.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:51 smithi148 ceph-mon[32020]: pgmap v221: 132 pgs: 1 activating, 131 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:26:51.939 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T09:26:51.940 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-17T09:26:51.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-17T09:26:52.149 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:26:53.544 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-mon[25650]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T09:26:53.544 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:53.544 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:53.544 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:53.544 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T09:26:53.544 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-mon[32020]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T09:26:53.544 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:53.544 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:53.545 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:53.545 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T09:26:53.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-mon[27402]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-17T09:26:53.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:53.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:53.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' 2024-09-17T09:26:53.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-17T09:26:53.774 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: ignoring --setuser ceph since I am not root 2024-09-17T09:26:53.774 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: ignoring --setgroup ceph since I am not root 2024-09-17T09:26:53.774 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-mgr[29602]: -- 172.21.15.157:0/2443728412 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fcec9e0d00 con 0x55fcec9b4800 2024-09-17T09:26:53.775 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:53.641+0000 7f9096727040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:26:53.775 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:53.703+0000 7f9096727040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:26:53.808 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ignoring --setuser ceph since I am not root 2024-09-17T09:26:53.808 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ignoring --setgroup ceph since I am not root 2024-09-17T09:26:53.808 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:53.638+0000 7f1573024040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T09:26:53.808 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:53.696+0000 7f1573024040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T09:26:54.035 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:53.857+0000 7f9096727040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:26:54.035 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:53 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:53.923+0000 7f9096727040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:26:54.167 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:53.855+0000 7f1573024040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T09:26:54.167 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:53 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:53.920+0000 7f1573024040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T09:26:54.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:54 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:54.072+0000 7f1573024040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:26:54.284 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:26:54.347 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:54 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:54.078+0000 7f9096727040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:26:54.347 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:54 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:54.217+0000 7f9096727040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:26:54.491 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:54 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:54.219+0000 7f1573024040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T09:26:54.697 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:54 smithi157 ceph-mon[27402]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T09:26:54.697 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:54 smithi157 ceph-mon[27402]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-17T09:26:54.697 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:54 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3361232271' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T09:26:54.698 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:54 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:54.529+0000 7f9096727040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:26:54.698 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:54 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:54.591+0000 7f9096727040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:26:54.806 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:54 smithi148 ceph-mon[25650]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T09:26:54.806 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:54 smithi148 ceph-mon[25650]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-17T09:26:54.806 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:54 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3361232271' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T09:26:54.806 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:54 smithi148 ceph-mon[32020]: from='mgr.14150 172.21.15.148:0/2744593793' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-17T09:26:54.807 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:54 smithi148 ceph-mon[32020]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-17T09:26:54.807 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:54 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3361232271' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-17T09:26:54.807 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:54 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:54.548+0000 7f1573024040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T09:26:54.807 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:54 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:54.609+0000 7f1573024040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T09:26:54.807 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:54 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:54.804+0000 7f1573024040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:26:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2879085245},{"type":"v1","addr":"172.21.15.148:6801","nonce":2879085245}]},"active_addr":"172.21.15.148:6801/2879085245","active_change":"2024-09-17T09:21:44.497370+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.148:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":4242182180}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":756526966}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":442080130}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":4139983843}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":2729952979}]}]} 2024-09-17T09:26:54.929 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-17T09:26:54.930 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-17T09:26:54.930 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T09:26:55.023 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:54 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:54.777+0000 7f9096727040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T09:26:55.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:55 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:55.724+0000 7f1573024040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:26:55.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:55 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:55.790+0000 7f1573024040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:26:55.987 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:55 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:55.709+0000 7f9096727040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T09:26:55.987 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:55 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:55.775+0000 7f9096727040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T09:26:56.635 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:56 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:56.515+0000 7f9096727040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:26:56.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:56 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:56.526+0000 7f1573024040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T09:26:56.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:56 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:56.647+0000 7f1573024040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:26:56.690 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:26:56.984 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:56 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:56.834+0000 7f1573024040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:26:56.984 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:56 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:56.917+0000 7f1573024040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:26:56.987 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:56 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:56.634+0000 7f9096727040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T09:26:56.987 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:56 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:56.777+0000 7f9096727040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T09:26:56.987 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:56 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:56.838+0000 7f9096727040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T09:26:56.987 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:56 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:56.985+0000 7f9096727040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:26:57.268 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:57 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:57.054+0000 7f9096727040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:26:57.268 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:57 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:57.113+0000 7f9096727040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:26:57.268 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:57 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:57.266+0000 7f9096727040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:26:57.377 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:57 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:57.076+0000 7f1573024040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T09:26:57.377 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:57 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:57.150+0000 7f1573024040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T09:26:57.377 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:57 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:57.215+0000 7f1573024040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T09:26:57.377 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:57 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:57.375+0000 7f1573024040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T09:26:57.523 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:57 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:57.331+0000 7f9096727040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:26:57.523 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:57 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:57.398+0000 7f9096727040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:26:57.523 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:57 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:57.479+0000 7f9096727040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:26:57.656 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:57 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:57.442+0000 7f1573024040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T09:26:57.656 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:57 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:57.509+0000 7f1573024040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T09:26:57.656 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:57 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:57.593+0000 7f1573024040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T09:26:57.656 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:57 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:57.654+0000 7f1573024040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:26:57.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:57 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:57.714+0000 7f1573024040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:26:58.023 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:57 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:57.538+0000 7f9096727040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T09:26:58.023 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:57 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:57.596+0000 7f9096727040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T09:26:58.706 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[32020]: Standby manager daemon x restarted 2024-09-17T09:26:58.706 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[32020]: Standby manager daemon x started 2024-09-17T09:26:58.707 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[32020]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:26:58.707 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[32020]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:26:58.707 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[32020]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:26:58.707 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[32020]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:26:58.707 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[25650]: Standby manager daemon x restarted 2024-09-17T09:26:58.707 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[25650]: Standby manager daemon x started 2024-09-17T09:26:58.707 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[25650]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:26:58.707 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[25650]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:26:58.707 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[25650]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:26:58.707 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-mon[25650]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:26:58.708 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:58.510+0000 7f1573024040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:26:58.708 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:26:58] ENGINE Bus STARTING 2024-09-17T09:26:58.708 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: CherryPy Checker: 2024-09-17T09:26:58.708 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: The Application mounted at '' has an empty config. 2024-09-17T09:26:58.708 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:26:58.708 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:26:58] ENGINE Serving on http://:::9283 2024-09-17T09:26:58.708 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:26:58 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:26:58] ENGINE Bus STARTED 2024-09-17T09:26:58.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-mon[27402]: Standby manager daemon x restarted 2024-09-17T09:26:58.725 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-mon[27402]: Standby manager daemon x started 2024-09-17T09:26:58.725 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-mon[27402]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-17T09:26:58.725 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-mon[27402]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-17T09:26:58.725 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-mon[27402]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-17T09:26:58.725 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-mon[27402]: from='mgr.? 172.21.15.157:0/1706730581' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-17T09:26:58.725 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:58.373+0000 7f9096727040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T09:26:58.725 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: [17/Sep/2024:09:26:58] ENGINE Bus STARTING 2024-09-17T09:26:58.725 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: CherryPy Checker: 2024-09-17T09:26:58.725 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: The Application mounted at '' has an empty config. 2024-09-17T09:26:58.726 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: 2024-09-17T09:26:58.726 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: [17/Sep/2024:09:26:58] ENGINE Serving on http://:::9283 2024-09-17T09:26:58.726 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:26:58 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x[29598]: [17/Sep/2024:09:26:58] ENGINE Bus STARTED 2024-09-17T09:26:59.363 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:26:59.363 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"fsid":"ff0bb856-74d5-11ef-bceb-c7b262605968","created":"2024-09-17T09:20:52.437233+0000","modified":"2024-09-17T09:26:58.512564+0000","last_up_change":"2024-09-17T09:26:01.156116+0000","last_in_change":"2024-09-17T09:25:41.855977+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T09:23:58.802065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-17T09:26:13.169434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-09-17T09:26:15.012928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"datapool","create_time":"2024-09-17T09:26:15.529320+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-17T09:26:17.005276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-17T09:26:19.053463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83fb0818-3f41-4cb5-915b-9833deef49d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6803","nonce":751433583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6805","nonce":751433583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6809","nonce":751433583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6807","nonce":751433583}]},"public_addr":"172.21.15.148:6803/751433583","cluster_addr":"172.21.15.148:6805/751433583","heartbeat_back_addr":"172.21.15.148:6809/751433583","heartbeat_front_addr":"172.21.15.148:6807/751433583","state":["exists","up"]},{"osd":1,"uuid":"9f13721c-e03d-4900-b493-158129624dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6811","nonce":3073615842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6813","nonce":3073615842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6817","nonce":3073615842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6815","nonce":3073615842}]},"public_addr":"172.21.15.148:6811/3073615842","cluster_addr":"172.21.15.148:6813/3073615842","heartbeat_back_addr":"172.21.15.148:6817/3073615842","heartbeat_front_addr":"172.21.15.148:6815/3073615842","state":["exists","up"]},{"osd":2,"uuid":"83fefc97-1359-4473-bbd5-00ec879dbd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6819","nonce":1077590729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6821","nonce":1077590729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6825","nonce":1077590729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6823","nonce":1077590729}]},"public_addr":"172.21.15.148:6819/1077590729","cluster_addr":"172.21.15.148:6821/1077590729","heartbeat_back_addr":"172.21.15.148:6825/1077590729","heartbeat_front_addr":"172.21.15.148:6823/1077590729","state":["exists","up"]},{"osd":3,"uuid":"129eb88b-a0a7-4f47-8f1f-8442bca041c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6827","nonce":3632250658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6829","nonce":3632250658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6833","nonce":3632250658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6831","nonce":3632250658}]},"public_addr":"172.21.15.148:6827/3632250658","cluster_addr":"172.21.15.148:6829/3632250658","heartbeat_back_addr":"172.21.15.148:6833/3632250658","heartbeat_front_addr":"172.21.15.148:6831/3632250658","state":["exists","up"]},{"osd":4,"uuid":"65795d19-b30b-4df4-a5df-2a02b7fdeddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6801","nonce":392952001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6803","nonce":392952001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6807","nonce":392952001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6805","nonce":392952001}]},"public_addr":"172.21.15.157:6801/392952001","cluster_addr":"172.21.15.157:6803/392952001","heartbeat_back_addr":"172.21.15.157:6807/392952001","heartbeat_front_addr":"172.21.15.157:6805/392952001","state":["exists","up"]},{"osd":5,"uuid":"90f62bba-d563-407d-8dc7-f0f0948b6eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6809","nonce":3770095390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6811","nonce":3770095390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6815","nonce":3770095390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6813","nonce":3770095390}]},"public_addr":"172.21.15.157:6809/3770095390","cluster_addr":"172.21.15.157:6811/3770095390","heartbeat_back_addr":"172.21.15.157:6815/3770095390","heartbeat_front_addr":"172.21.15.157:6813/3770095390","state":["exists","up"]},{"osd":6,"uuid":"17521d9e-6ccd-4bd1-a6a6-04b2f193f262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6817","nonce":3080568127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6819","nonce":3080568127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6823","nonce":3080568127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6821","nonce":3080568127}]},"public_addr":"172.21.15.157:6817/3080568127","cluster_addr":"172.21.15.157:6819/3080568127","heartbeat_back_addr":"172.21.15.157:6823/3080568127","heartbeat_front_addr":"172.21.15.157:6821/3080568127","state":["exists","up"]},{"osd":7,"uuid":"4109e289-2bc0-414e-8cf0-24acaf243f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6825","nonce":1756902340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6827","nonce":1756902340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6831","nonce":1756902340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6829","nonce":1756902340}]},"public_addr":"172.21.15.157:6825/1756902340","cluster_addr":"172.21.15.157:6827/1756902340","heartbeat_back_addr":"172.21.15.157:6831/1756902340","heartbeat_front_addr":"172.21.15.157:6829/1756902340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:06.820629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:30.276799+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:55.775977+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:21.248005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:45.557104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:25:08.921480+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:25:33.373493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:25:59.484247+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.3","mappings":[{"from":7,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:0/4139983843":"2024-09-18T09:26:58.512530+0000","172.21.15.148:6800/2879085245":"2024-09-18T09:26:58.512530+0000","172.21.15.148:6800/3242443044":"2024-09-18T09:21:22.694776+0000","172.21.15.148:6800/2921533551":"2024-09-18T09:21:44.497259+0000","172.21.15.148:6801/3242443044":"2024-09-18T09:21:22.694776+0000","172.21.15.148:6801/2879085245":"2024-09-18T09:26:58.512530+0000","172.21.15.148:0/270732129":"2024-09-18T09:21:22.694776+0000","172.21.15.148:6801/2921533551":"2024-09-18T09:21:44.497259+0000","172.21.15.148:0/2737221819":"2024-09-18T09:21:44.497259+0000","172.21.15.148:0/2186092680":"2024-09-18T09:21:22.694776+0000","172.21.15.148:0/517099512":"2024-09-18T09:21:22.694776+0000","172.21.15.148:0/3221867330":"2024-09-18T09:21:44.497259+0000","172.21.15.148:0/2957040738":"2024-09-18T09:21:44.497259+0000","172.21.15.148:0/2729952979":"2024-09-18T09:26:58.512530+0000","172.21.15.148:0/756526966":"2024-09-18T09:26:58.512530+0000","172.21.15.148:0/442080130":"2024-09-18T09:26:58.512530+0000","172.21.15.148:0/4242182180":"2024-09-18T09:26:58.512530+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T09:26:59.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: Active manager daemon y restarted 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: Activating manager daemon y 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: mgrmap e19: y(active, starting, since 0.00720249s), standbys: x 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:26:59.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: Manager daemon y is now available 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:59.669 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:59.670 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]': finished 2024-09-17T09:26:59.670 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]': finished 2024-09-17T09:26:59.670 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:26:59.670 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:59.670 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:59.670 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:26:59.670 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:26:59.670 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:26:59.670 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/2032917308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:26:59.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: Active manager daemon y restarted 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: Activating manager daemon y 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: mgrmap e19: y(active, starting, since 0.00720249s), standbys: x 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:26:59.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: Manager daemon y is now available 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]': finished 2024-09-17T09:26:59.672 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]': finished 2024-09-17T09:26:59.673 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:26:59.673 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:59.673 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:59.673 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:26:59.673 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:26:59.673 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:26:59.673 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:26:59 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/2032917308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:26:59.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: Active manager daemon y restarted 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: Activating manager daemon y 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: osdmap e71: 8 total, 8 up, 8 in 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: mgrmap e19: y(active, starting, since 0.00720249s), standbys: x 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-17T09:26:59.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: Manager daemon y is now available 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:59.775 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-09-17T09:26:59.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]': finished 2024-09-17T09:26:59.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]': finished 2024-09-17T09:26:59.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:26:59.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:26:59.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:26:59.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:26:59.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-17T09:26:59.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-17T09:26:59.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:26:59 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/2032917308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:27:00.218 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-17T09:27:00.219 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-17T09:27:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:00.519 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ::ffff:172.21.15.157 - - [17/Sep/2024:09:27:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:27:00.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-17T09:27:00.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-17T09:27:00.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: Updating smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: Updating smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: mgrmap e20: y(active, since 1.01093s), standbys: x 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:00 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.807 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-17T09:27:00.807 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: Updating smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: Updating smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: mgrmap e20: y(active, since 1.01093s), standbys: x 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.808 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.809 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-17T09:27:00.809 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: Updating smithi157:/etc/ceph/ceph.conf 2024-09-17T09:27:00.809 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: Updating smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:27:00.809 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: Updating smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-17T09:27:00.809 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:27:00.809 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-17T09:27:00.809 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: mgrmap e20: y(active, since 1.01093s), standbys: x 2024-09-17T09:27:00.809 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:00.809 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:00.809 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.809 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.810 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.810 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.810 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:00.810 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:01.167 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:27:00.841+0000 7f14c73cb640 -1 log_channel(cephadm) log [ERR] : [17/Sep/2024:09:27:00] ENGINE Error in HTTPServer.serve 2024-09-17T09:27:01.167 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: Traceback (most recent call last): 2024-09-17T09:27:01.167 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self._connections.run(self.expiration_interval) 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self._run(expiration_interval) 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: s = self.context.wrap_socket( 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: return self.sslsocket_class._create( 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:27:01.168 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self.do_handshake() 2024-09-17T09:27:01.169 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:27:01.169 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: self._sslobj.do_handshake() 2024-09-17T09:27:01.169 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:27:01.169 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:27:01.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:01 smithi148 ceph-mon[25650]: Updating smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:27:01.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:01 smithi148 ceph-mon[25650]: Updating smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:27:01.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:01 smithi148 ceph-mon[25650]: Deploying daemon alertmanager.a on smithi148 2024-09-17T09:27:01.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:01 smithi148 ceph-mon[25650]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:01.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:01 smithi148 ceph-mon[25650]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-17T09:27:01.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:01 smithi148 ceph-mon[32020]: Updating smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:27:01.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:01 smithi148 ceph-mon[32020]: Updating smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:27:01.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:01 smithi148 ceph-mon[32020]: Deploying daemon alertmanager.a on smithi148 2024-09-17T09:27:01.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:01 smithi148 ceph-mon[32020]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:01.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:01 smithi148 ceph-mon[32020]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-17T09:27:02.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:01 smithi157 ceph-mon[27402]: Updating smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:27:02.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:01 smithi157 ceph-mon[27402]: Updating smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-17T09:27:02.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:01 smithi157 ceph-mon[27402]: Deploying daemon alertmanager.a on smithi148 2024-09-17T09:27:02.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:01 smithi157 ceph-mon[27402]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:02.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:01 smithi157 ceph-mon[27402]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-17T09:27:02.686 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:02 smithi148 ceph-mon[25650]: [17/Sep/2024:09:27:00] ENGINE Bus STARTING 2024-09-17T09:27:02.686 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:02 smithi148 ceph-mon[25650]: [17/Sep/2024:09:27:00] ENGINE Serving on http://172.21.15.148:8765 2024-09-17T09:27:02.686 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:02 smithi148 ceph-mon[25650]: [17/Sep/2024:09:27:00] ENGINE Serving on https://172.21.15.148:7151 2024-09-17T09:27:02.686 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:02 smithi148 ceph-mon[25650]: [17/Sep/2024:09:27:00] ENGINE Bus STARTED 2024-09-17T09:27:02.686 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:02 smithi148 ceph-mon[25650]: [17/Sep/2024:09:27:00] ENGINE Error in HTTPServer.serve 2024-09-17T09:27:02.686 INFO:journalctl@ceph.mon.a.smithi148.stdout: Traceback (most recent call last): 2024-09-17T09:27:02.686 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:27:02.687 INFO:journalctl@ceph.mon.a.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-09-17T09:27:02.687 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:27:02.687 INFO:journalctl@ceph.mon.a.smithi148.stdout: self._run(expiration_interval) 2024-09-17T09:27:02.687 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:27:02.687 INFO:journalctl@ceph.mon.a.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:27:02.687 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:27:02.688 INFO:journalctl@ceph.mon.a.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:27:02.688 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:27:02.688 INFO:journalctl@ceph.mon.a.smithi148.stdout: s = self.context.wrap_socket( 2024-09-17T09:27:02.688 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:27:02.688 INFO:journalctl@ceph.mon.a.smithi148.stdout: return self.sslsocket_class._create( 2024-09-17T09:27:02.688 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:27:02.688 INFO:journalctl@ceph.mon.a.smithi148.stdout: self.do_handshake() 2024-09-17T09:27:02.689 INFO:journalctl@ceph.mon.a.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:27:02.689 INFO:journalctl@ceph.mon.a.smithi148.stdout: self._sslobj.do_handshake() 2024-09-17T09:27:02.689 INFO:journalctl@ceph.mon.a.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:27:02.689 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:02 smithi148 ceph-mon[32020]: [17/Sep/2024:09:27:00] ENGINE Bus STARTING 2024-09-17T09:27:02.689 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:02 smithi148 ceph-mon[32020]: [17/Sep/2024:09:27:00] ENGINE Serving on http://172.21.15.148:8765 2024-09-17T09:27:02.689 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:02 smithi148 ceph-mon[32020]: [17/Sep/2024:09:27:00] ENGINE Serving on https://172.21.15.148:7151 2024-09-17T09:27:02.689 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:02 smithi148 ceph-mon[32020]: [17/Sep/2024:09:27:00] ENGINE Bus STARTED 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:02 smithi148 ceph-mon[32020]: [17/Sep/2024:09:27:00] ENGINE Error in HTTPServer.serve 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: Traceback (most recent call last): 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: self._connections.run(self.expiration_interval) 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: self._run(expiration_interval) 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: s = self.context.wrap_socket( 2024-09-17T09:27:02.690 INFO:journalctl@ceph.mon.c.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:27:02.691 INFO:journalctl@ceph.mon.c.smithi148.stdout: return self.sslsocket_class._create( 2024-09-17T09:27:02.691 INFO:journalctl@ceph.mon.c.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:27:02.691 INFO:journalctl@ceph.mon.c.smithi148.stdout: self.do_handshake() 2024-09-17T09:27:02.691 INFO:journalctl@ceph.mon.c.smithi148.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:27:02.691 INFO:journalctl@ceph.mon.c.smithi148.stdout: self._sslobj.do_handshake() 2024-09-17T09:27:02.691 INFO:journalctl@ceph.mon.c.smithi148.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:27:03.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:02 smithi157 ceph-mon[27402]: [17/Sep/2024:09:27:00] ENGINE Bus STARTING 2024-09-17T09:27:03.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:02 smithi157 ceph-mon[27402]: [17/Sep/2024:09:27:00] ENGINE Serving on http://172.21.15.148:8765 2024-09-17T09:27:03.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:02 smithi157 ceph-mon[27402]: [17/Sep/2024:09:27:00] ENGINE Serving on https://172.21.15.148:7151 2024-09-17T09:27:03.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:02 smithi157 ceph-mon[27402]: [17/Sep/2024:09:27:00] ENGINE Bus STARTED 2024-09-17T09:27:03.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:02 smithi157 ceph-mon[27402]: [17/Sep/2024:09:27:00] ENGINE Error in HTTPServer.serve 2024-09-17T09:27:03.024 INFO:journalctl@ceph.mon.b.smithi157.stdout: Traceback (most recent call last): 2024-09-17T09:27:03.024 INFO:journalctl@ceph.mon.b.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-17T09:27:03.024 INFO:journalctl@ceph.mon.b.smithi157.stdout: self._connections.run(self.expiration_interval) 2024-09-17T09:27:03.024 INFO:journalctl@ceph.mon.b.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-17T09:27:03.024 INFO:journalctl@ceph.mon.b.smithi157.stdout: self._run(expiration_interval) 2024-09-17T09:27:03.024 INFO:journalctl@ceph.mon.b.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-17T09:27:03.024 INFO:journalctl@ceph.mon.b.smithi157.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-17T09:27:03.024 INFO:journalctl@ceph.mon.b.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-17T09:27:03.025 INFO:journalctl@ceph.mon.b.smithi157.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-17T09:27:03.025 INFO:journalctl@ceph.mon.b.smithi157.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-17T09:27:03.025 INFO:journalctl@ceph.mon.b.smithi157.stdout: s = self.context.wrap_socket( 2024-09-17T09:27:03.025 INFO:journalctl@ceph.mon.b.smithi157.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-17T09:27:03.025 INFO:journalctl@ceph.mon.b.smithi157.stdout: return self.sslsocket_class._create( 2024-09-17T09:27:03.025 INFO:journalctl@ceph.mon.b.smithi157.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-17T09:27:03.025 INFO:journalctl@ceph.mon.b.smithi157.stdout: self.do_handshake() 2024-09-17T09:27:03.025 INFO:journalctl@ceph.mon.b.smithi157.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-17T09:27:03.025 INFO:journalctl@ceph.mon.b.smithi157.stdout: self._sslobj.do_handshake() 2024-09-17T09:27:03.025 INFO:journalctl@ceph.mon.b.smithi157.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-17T09:27:03.077 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:27:03.077 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"fsid":"ff0bb856-74d5-11ef-bceb-c7b262605968","created":"2024-09-17T09:20:52.437233+0000","modified":"2024-09-17T09:26:58.512564+0000","last_up_change":"2024-09-17T09:26:01.156116+0000","last_in_change":"2024-09-17T09:25:41.855977+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T09:23:58.802065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-17T09:26:13.169434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-09-17T09:26:15.012928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"datapool","create_time":"2024-09-17T09:26:15.529320+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-17T09:26:17.005276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-17T09:26:19.053463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83fb0818-3f41-4cb5-915b-9833deef49d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6803","nonce":751433583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6805","nonce":751433583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6809","nonce":751433583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":751433583},{"type":"v1","addr":"172.21.15.148:6807","nonce":751433583}]},"public_addr":"172.21.15.148:6803/751433583","cluster_addr":"172.21.15.148:6805/751433583","heartbeat_back_addr":"172.21.15.148:6809/751433583","heartbeat_front_addr":"172.21.15.148:6807/751433583","state":["exists","up"]},{"osd":1,"uuid":"9f13721c-e03d-4900-b493-158129624dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6811","nonce":3073615842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6813","nonce":3073615842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6817","nonce":3073615842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":3073615842},{"type":"v1","addr":"172.21.15.148:6815","nonce":3073615842}]},"public_addr":"172.21.15.148:6811/3073615842","cluster_addr":"172.21.15.148:6813/3073615842","heartbeat_back_addr":"172.21.15.148:6817/3073615842","heartbeat_front_addr":"172.21.15.148:6815/3073615842","state":["exists","up"]},{"osd":2,"uuid":"83fefc97-1359-4473-bbd5-00ec879dbd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6819","nonce":1077590729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6821","nonce":1077590729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6825","nonce":1077590729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1077590729},{"type":"v1","addr":"172.21.15.148:6823","nonce":1077590729}]},"public_addr":"172.21.15.148:6819/1077590729","cluster_addr":"172.21.15.148:6821/1077590729","heartbeat_back_addr":"172.21.15.148:6825/1077590729","heartbeat_front_addr":"172.21.15.148:6823/1077590729","state":["exists","up"]},{"osd":3,"uuid":"129eb88b-a0a7-4f47-8f1f-8442bca041c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6827","nonce":3632250658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6829","nonce":3632250658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6833","nonce":3632250658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3632250658},{"type":"v1","addr":"172.21.15.148:6831","nonce":3632250658}]},"public_addr":"172.21.15.148:6827/3632250658","cluster_addr":"172.21.15.148:6829/3632250658","heartbeat_back_addr":"172.21.15.148:6833/3632250658","heartbeat_front_addr":"172.21.15.148:6831/3632250658","state":["exists","up"]},{"osd":4,"uuid":"65795d19-b30b-4df4-a5df-2a02b7fdeddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6801","nonce":392952001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6803","nonce":392952001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6807","nonce":392952001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":392952001},{"type":"v1","addr":"172.21.15.157:6805","nonce":392952001}]},"public_addr":"172.21.15.157:6801/392952001","cluster_addr":"172.21.15.157:6803/392952001","heartbeat_back_addr":"172.21.15.157:6807/392952001","heartbeat_front_addr":"172.21.15.157:6805/392952001","state":["exists","up"]},{"osd":5,"uuid":"90f62bba-d563-407d-8dc7-f0f0948b6eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6809","nonce":3770095390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6811","nonce":3770095390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6815","nonce":3770095390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3770095390},{"type":"v1","addr":"172.21.15.157:6813","nonce":3770095390}]},"public_addr":"172.21.15.157:6809/3770095390","cluster_addr":"172.21.15.157:6811/3770095390","heartbeat_back_addr":"172.21.15.157:6815/3770095390","heartbeat_front_addr":"172.21.15.157:6813/3770095390","state":["exists","up"]},{"osd":6,"uuid":"17521d9e-6ccd-4bd1-a6a6-04b2f193f262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6817","nonce":3080568127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6819","nonce":3080568127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6823","nonce":3080568127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3080568127},{"type":"v1","addr":"172.21.15.157:6821","nonce":3080568127}]},"public_addr":"172.21.15.157:6817/3080568127","cluster_addr":"172.21.15.157:6819/3080568127","heartbeat_back_addr":"172.21.15.157:6823/3080568127","heartbeat_front_addr":"172.21.15.157:6821/3080568127","state":["exists","up"]},{"osd":7,"uuid":"4109e289-2bc0-414e-8cf0-24acaf243f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6825","nonce":1756902340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6827","nonce":1756902340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6831","nonce":1756902340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":1756902340},{"type":"v1","addr":"172.21.15.157:6829","nonce":1756902340}]},"public_addr":"172.21.15.157:6825/1756902340","cluster_addr":"172.21.15.157:6827/1756902340","heartbeat_back_addr":"172.21.15.157:6831/1756902340","heartbeat_front_addr":"172.21.15.157:6829/1756902340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:06.820629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:30.276799+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:23:55.775977+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:21.248005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:24:45.557104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:25:08.921480+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:25:33.373493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T09:25:59.484247+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.3","mappings":[{"from":7,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:0/4139983843":"2024-09-18T09:26:58.512530+0000","172.21.15.148:6800/2879085245":"2024-09-18T09:26:58.512530+0000","172.21.15.148:6800/3242443044":"2024-09-18T09:21:22.694776+0000","172.21.15.148:6800/2921533551":"2024-09-18T09:21:44.497259+0000","172.21.15.148:6801/3242443044":"2024-09-18T09:21:22.694776+0000","172.21.15.148:6801/2879085245":"2024-09-18T09:26:58.512530+0000","172.21.15.148:0/270732129":"2024-09-18T09:21:22.694776+0000","172.21.15.148:6801/2921533551":"2024-09-18T09:21:44.497259+0000","172.21.15.148:0/2737221819":"2024-09-18T09:21:44.497259+0000","172.21.15.148:0/2186092680":"2024-09-18T09:21:22.694776+0000","172.21.15.148:0/517099512":"2024-09-18T09:21:22.694776+0000","172.21.15.148:0/3221867330":"2024-09-18T09:21:44.497259+0000","172.21.15.148:0/2957040738":"2024-09-18T09:21:44.497259+0000","172.21.15.148:0/2729952979":"2024-09-18T09:26:58.512530+0000","172.21.15.148:0/756526966":"2024-09-18T09:26:58.512530+0000","172.21.15.148:0/442080130":"2024-09-18T09:26:58.512530+0000","172.21.15.148:0/4242182180":"2024-09-18T09:26:58.512530+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T09:27:03.806 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:03 smithi148 ceph-mon[25650]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:03.806 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:03 smithi148 ceph-mon[25650]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-17T09:27:03.806 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:03 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/2938275388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:27:03.806 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:03 smithi148 ceph-mon[32020]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:03.806 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:03 smithi148 ceph-mon[32020]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-17T09:27:03.806 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:03 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/2938275388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:27:04.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:03 smithi157 ceph-mon[27402]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:04.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:03 smithi157 ceph-mon[27402]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-17T09:27:04.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:03 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/2938275388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-17T09:27:04.282 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-17T09:27:04.283 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-17T09:27:04.283 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-17T09:27:04.283 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-17T09:27:04.284 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-17T09:27:04.284 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-17T09:27:04.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-17T09:27:04.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-17T09:27:05.867 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:05 smithi148 ceph-mon[25650]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:05.867 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:05 smithi148 ceph-mon[32020]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:06.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:05 smithi157 ceph-mon[27402]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-17T09:27:07.616 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:07.616 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:07.629 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:07.630 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:07.633 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:07.694 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:07.700 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:07.755 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:07 smithi148 ceph-mon[25650]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-17T09:27:07.755 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:07 smithi148 ceph-mon[32020]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-17T09:27:08.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:07 smithi157 ceph-mon[27402]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-17T09:27:09.856 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:09 smithi148 ceph-mon[25650]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T09:27:09.857 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:09 smithi148 ceph-mon[32020]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T09:27:10.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:27:09 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:27:10.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:09 smithi157 ceph-mon[27402]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-17T09:27:10.554 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:10 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ::ffff:172.21.15.157 - - [17/Sep/2024:09:27:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:27:10.831 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:10 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:10.896 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:10 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:10.896 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:10 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:11.801 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:11 smithi148 ceph-mon[25650]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-17T09:27:11.802 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:11 smithi148 ceph-mon[32020]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-17T09:27:11.904 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:11 smithi157 ceph-mon[27402]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-17T09:27:12.741 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:12 smithi148 ceph-mon[25650]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:27:12.741 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:12 smithi148 ceph-mon[32020]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:27:12.902 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:12 smithi157 ceph-mon[27402]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:27:13.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:13 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:27:13.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:13 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:27:14.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:13 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:27:14.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:14 smithi148 ceph-mon[25650]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:27:14.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:14 smithi148 ceph-mon[32020]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:27:15.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:14 smithi157 ceph-mon[27402]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:27:15.418 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:27:15 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[55656]: ts=2024-09-17T09:27:15.386Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-17T09:27:15.758 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:15 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:27:15] ENGINE Bus STOPPING 2024-09-17T09:27:15.758 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:27:15 smithi148 bash[55547]: 645b209ad43dcef18d66838374808a8d0777e767c4a82c2684b3714fbb6b95a4 2024-09-17T09:27:15.758 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:27:15 smithi148 systemd[1]: Started Ceph alertmanager.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:27:16.023 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:15 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:27:15] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T09:27:16.024 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:15 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:27:15] ENGINE Bus STOPPED 2024-09-17T09:27:16.024 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:15 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:27:15] ENGINE Bus STARTING 2024-09-17T09:27:16.024 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:27:16] ENGINE Serving on http://:::9283 2024-09-17T09:27:16.296 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:27:16] ENGINE Bus STARTED 2024-09-17T09:27:16.650 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.650 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.650 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.650 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.650 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[32020]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T09:27:16.650 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.650 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.651 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:27:16.651 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[25650]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T09:27:16.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:27:16.651 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:16 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:16 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:16 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:16 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:16 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:16 smithi157 ceph-mon[27402]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-17T09:27:16.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:16 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:16 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:16.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:16 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:27:16.774 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:16 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:17.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:17 smithi148 ceph-mon[32020]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:27:17.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:17 smithi148 ceph-mon[32020]: Deploying daemon grafana.a on smithi157 2024-09-17T09:27:17.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:17 smithi148 ceph-mon[32020]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:27:17.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:17 smithi148 ceph-mon[25650]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:27:17.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:17 smithi148 ceph-mon[25650]: Deploying daemon grafana.a on smithi157 2024-09-17T09:27:17.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:17 smithi148 ceph-mon[25650]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:27:17.668 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:27:17 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[55656]: ts=2024-09-17T09:27:17.356Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000153634s 2024-09-17T09:27:17.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:17 smithi157 ceph-mon[27402]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-17T09:27:17.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:17 smithi157 ceph-mon[27402]: Deploying daemon grafana.a on smithi157 2024-09-17T09:27:17.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:17 smithi157 ceph-mon[27402]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-17T09:27:18.606 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:18 smithi148 ceph-mon[25650]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:18.606 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:18 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:18.606 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:18 smithi148 ceph-mon[32020]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:18.607 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:18 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:19.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:18 smithi157 ceph-mon[27402]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:19.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:18 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:19.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:19 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:19.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:19 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:20.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:19 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:20.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:27:19 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:27:20.609 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:20 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ::ffff:172.21.15.157 - - [17/Sep/2024:09:27:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:27:20.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:20 smithi148 ceph-mon[25650]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:20.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:20 smithi148 ceph-mon[32020]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:21.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:20 smithi157 ceph-mon[27402]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:21.150 INFO:teuthology.orchestra.run.smithi148.stdout:197568495638 2024-09-17T09:27:21.151 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-17T09:27:21.651 INFO:teuthology.orchestra.run.smithi148.stdout:128849018913 2024-09-17T09:27:21.651 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-17T09:27:22.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:22 smithi148 ceph-mon[25650]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:22.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:22 smithi148 ceph-mon[32020]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:23.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:22 smithi157 ceph-mon[27402]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:23.353 INFO:teuthology.orchestra.run.smithi148.stdout:34359738419 2024-09-17T09:27:23.353 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-17T09:27:23.877 INFO:teuthology.orchestra.run.smithi148.stdout:231928234002 2024-09-17T09:27:23.878 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-17T09:27:24.535 INFO:teuthology.orchestra.run.smithi148.stdout:163208757277 2024-09-17T09:27:24.536 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-17T09:27:24.819 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:24 smithi157 ceph-mon[27402]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:24.889 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:24 smithi148 ceph-mon[32020]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:24.889 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:24 smithi148 ceph-mon[25650]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:25.658 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:27:25 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[55656]: ts=2024-09-17T09:27:25.359Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003208406s 2024-09-17T09:27:26.699 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-mon[27402]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:26.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:26 smithi148 ceph-mon[25650]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:26.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:26 smithi148 ceph-mon[32020]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:26.949 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.948471848Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-17T09:27:26.950 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.949100593Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-17T09:27:26.970 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:55834574896 2024-09-17T09:27:26.988 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-17T09:27:27.209 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.949701527Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-17T09:27:27.209 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.949993498Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-17T09:27:27.209 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.950256857Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.950603696Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.951011849Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.951410796Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.951772124Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.952208033Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.95288999Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.953513595Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.953792065Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.954228222Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.954622327Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.955264885Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.955566542Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.955937129Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-17T09:27:27.210 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.956348958Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-17T09:27:27.211 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.956604561Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-17T09:27:27.212 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.957050905Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-17T09:27:27.212 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.957483593Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-17T09:27:27.212 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.957891745Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-17T09:27:27.212 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.958345834Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-17T09:27:27.212 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.958792787Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-17T09:27:27.212 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.959244113Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.959530688Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.960200709Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.960598808Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.961252025Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.96163771Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.962050913Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.962447664Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.964004644Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.96438298Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.964799326Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.965124307Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.96546799Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-17T09:27:27.213 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.96573783Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.966401637Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.967061696Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.967478371Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.967884769Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.968201034Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.968858523Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.969291347Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.969592054Z level=info msg="Executing migration" id="Move region to single row" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.969909415Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.970339017Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.970752665Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.971184055Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.971591546Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-17T09:27:27.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.971956948Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-17T09:27:27.215 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.972366851Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-17T09:27:27.215 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.972643756Z level=info msg="Executing migration" id="create test_data table" 2024-09-17T09:27:27.215 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.973132999Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-17T09:27:27.215 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.973517065Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-17T09:27:27.215 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.973921018Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-17T09:27:27.215 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.974335353Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-17T09:27:27.215 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.974603355Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-17T09:27:27.215 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.974884172Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-17T09:27:27.215 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.975192658Z level=info msg="Executing migration" id="create team table" 2024-09-17T09:27:27.215 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.975571317Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-17T09:27:27.216 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.976000845Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-17T09:27:27.216 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.976420478Z level=info msg="Executing migration" id="create team member table" 2024-09-17T09:27:27.216 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.976799318Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-17T09:27:27.216 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.977220522Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-17T09:27:27.216 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.97763908Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-17T09:27:27.216 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.978069126Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-17T09:27:27.216 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.978798355Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-17T09:27:27.216 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.979490344Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-17T09:27:27.216 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.98019438Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.980585471Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.981037195Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.981461838Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.981894068Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.982379045Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.982802213Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.98322836Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.98363979Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.984014641Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.984349856Z level=info msg="Executing migration" id="create tag table" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.984732414Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.985161803Z level=info msg="Executing migration" id="create login attempt table" 2024-09-17T09:27:27.217 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.98554704Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-17T09:27:27.218 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.98600797Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-17T09:27:27.218 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.986417852Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-17T09:27:27.218 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.988279818Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-17T09:27:27.218 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.988656645Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-17T09:27:27.218 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.98908672Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-17T09:27:27.218 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.989399869Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-17T09:27:27.218 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.98972204Z level=info msg="Executing migration" id="create user auth table" 2024-09-17T09:27:27.218 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.990125062Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-17T09:27:27.218 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.990537506Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-17T09:27:27.218 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.990795387Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-17T09:27:27.218 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.991517675Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-17T09:27:27.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.992252606Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-17T09:27:27.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.992978703Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-17T09:27:27.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.993736896Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-17T09:27:27.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.994201089Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-17T09:27:27.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.99491877Z level=info msg="Executing migration" id="create server_lock table" 2024-09-17T09:27:27.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.995308537Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-17T09:27:27.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.995730801Z level=info msg="Executing migration" id="create user auth token table" 2024-09-17T09:27:27.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.996158453Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-17T09:27:27.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.996577171Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-17T09:27:27.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.996997463Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-17T09:27:27.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.997414082Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.998181798Z level=info msg="Executing migration" id="create cache_data table" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.998583404Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.999039395Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.999471062Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:26.999867899Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.000171806Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.000437255Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.000837702Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.00123767Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.001634395Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.001862195Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.002248041Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.002630258Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-17T09:27:27.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.003000848Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.003396578Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.004166498Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.004555259Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.004799126Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.005193173Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.005602983Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.006014197Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.0063099Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.006703054Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.007151546Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.007617387Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.008049102Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-17T09:27:27.222 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.00881549Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.009240858Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.009645783Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.013040749Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.016392986Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.016831124Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.017260076Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.018083746Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.01851314Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.018972332Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.019405948Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.019869379Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.020176116Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-17T09:27:27.223 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.020976052Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.021786337Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.022613929Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.023028437Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.023453394Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.024267481Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.025078217Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.025517034Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.026329385Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.027153571Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.02745875Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.027912003Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.028362316Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.028797636Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-17T09:27:27.224 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.029095609Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.030041927Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.030862171Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.031682356Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.032505299Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.033328436Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.033620405Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.034002416Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.034815391Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.035115787Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.036011729Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.036441017Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.037271077Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-17T09:27:27.225 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.037656231Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.038089579Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.038929042Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.039324866Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.039773395Z level=info msg="Executing migration" id="create alert_image table" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.040181562Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.040609079Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.040919839Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.04133174Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.041733753Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.041830779Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.042110769Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.042462049Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.042864921Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-17T09:27:27.226 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.043030971Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.043191035Z level=info msg="alerts found to migrate" alerts=0 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.043608814Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.044057977Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.044492043Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.044884553Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.045357895Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.045809376Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.046127758Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.046521958Z level=info msg="Executing migration" id="create data_keys table" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.046933652Z level=info msg="Executing migration" id="create secrets table" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.047412187Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.051736859Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.052673009Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.052986269Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-17T09:27:27.227 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.057234828Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.061473592Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.061890944Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.06236283Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.062677446Z level=info msg="Executing migration" id="create permission table" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.063079489Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.063532597Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.063972332Z level=info msg="Executing migration" id="create role table" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.064390615Z level=info msg="Executing migration" id="add column display_name" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.065313432Z level=info msg="Executing migration" id="add column group_name" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.066227293Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.066666227Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.067101568Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.067537402Z level=info msg="Executing migration" id="create team role table" 2024-09-17T09:27:27.228 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.067924636Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.068444958Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.06890757Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.069368601Z level=info msg="Executing migration" id="create user role table" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.069757846Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.070198912Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.070647706Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.071095426Z level=info msg="Executing migration" id="create builtin role table" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.07150311Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.071927185Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.072384835Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.073402733Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.073881695Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.074331048Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-17T09:27:27.229 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.074751643Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.07518774Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.075570854Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.076022838Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.076998027Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.077423514Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.077881951Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.078202574Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.078591293Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.079056422Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.079420018Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.079697239Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.080076899Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.080463517Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-17T09:27:27.230 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.080924088Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.081916336Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.082240079Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.082649812Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.083097395Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.083709521Z level=info msg="Executing migration" id="add correlation config column" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.08469976Z level=info msg="Executing migration" id="create entity_events table" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.085155549Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.085619323Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.085719275Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.086021974Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.086128344Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.086394317Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-17T09:27:27.231 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.08674338Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.087163076Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.087593099Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.088038368Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.088465635Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.088858621Z level=info msg="Executing migration" id="Drop public config table" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.089285687Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.089708238Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.090152823Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.090623393Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.091065006Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.094045073Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.095034026Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.096043229Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-17T09:27:27.232 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.096363938Z level=info msg="Executing migration" id="add share column" 2024-09-17T09:27:27.233 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.097351981Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-17T09:27:27.233 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.097783182Z level=info msg="Executing migration" id="create file table" 2024-09-17T09:27:27.233 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.098224664Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-17T09:27:27.233 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.098665985Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-17T09:27:27.233 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.099140421Z level=info msg="Executing migration" id="create file_meta table" 2024-09-17T09:27:27.233 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.09952339Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-17T09:27:27.233 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.099982676Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-17T09:27:27.233 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.100277977Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-17T09:27:27.233 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.100609531Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-17T09:27:27.233 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.100884807Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-17T09:27:27.234 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.101323241Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-17T09:27:27.234 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.10235772Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-17T09:27:27.234 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.102662994Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-17T09:27:27.234 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.103107057Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-17T09:27:27.234 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.103487085Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-17T09:27:27.234 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.103766002Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-17T09:27:27.234 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.104181771Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-17T09:27:27.234 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.105187594Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-17T09:27:27.234 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.106203675Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-17T09:27:27.234 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.106621468Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-17T09:27:27.234 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.116384256Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.116841128Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.117290967Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.120694587Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.121020854Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.12135959Z level=info msg="Executing migration" id="create folder table" 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.121716539Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.122161695Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.122595229Z level=info msg="Executing migration" id="Update folder title length" 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.122845785Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=migrator t=2024-09-17T09:27:27.12327782Z level=info msg="migrations completed" performed=485 skipped=0 duration=250.218214ms 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=sqlstore t=2024-09-17T09:27:27.12369209Z level=info msg="Created default organization" 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=plugin.loader t=2024-09-17T09:27:27.180357999Z level=info msg="Plugin registered" pluginID=input 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=plugin.loader t=2024-09-17T09:27:27.208289745Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=plugin.loader t=2024-09-17T09:27:27.208332562Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=secrets t=2024-09-17T09:27:27.208369614Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-17T09:27:27.235 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=query_data t=2024-09-17T09:27:27.212551492Z level=info msg="Query Service initialization" 2024-09-17T09:27:27.236 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=live.push_http t=2024-09-17T09:27:27.215333747Z level=info msg="Live Push Gateway initialization" 2024-09-17T09:27:28.029 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.030 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.030 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.030 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.030 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-mon[27402]: Deploying daemon node-exporter.a on smithi148 2024-09-17T09:27:28.030 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=infra.usagestats.collector t=2024-09-17T09:27:27.973303174Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-17T09:27:28.030 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=provisioning.datasources t=2024-09-17T09:27:28.028792239Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-17T09:27:28.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:27 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:27 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:27 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:27 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:27 smithi148 ceph-mon[25650]: Deploying daemon node-exporter.a on smithi148 2024-09-17T09:27:28.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:27 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:27 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:27 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:27 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:28.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:27 smithi148 ceph-mon[32020]: Deploying daemon node-exporter.a on smithi148 2024-09-17T09:27:28.508 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=provisioning.datasources t=2024-09-17T09:27:28.061082201Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-17T09:27:28.509 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=provisioning.alerting t=2024-09-17T09:27:28.08301034Z level=info msg="starting to provision alerting" 2024-09-17T09:27:28.509 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=provisioning.alerting t=2024-09-17T09:27:28.083032751Z level=info msg="finished to provision alerting" 2024-09-17T09:27:28.509 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=grafanaStorageLogger t=2024-09-17T09:27:28.083169231Z level=info msg="storage starting" 2024-09-17T09:27:28.509 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=ngalert.state.manager t=2024-09-17T09:27:28.083219216Z level=info msg="Warming state cache for startup" 2024-09-17T09:27:28.509 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=http.server t=2024-09-17T09:27:28.085034151Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-17T09:27:28.509 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=ngalert.state.manager t=2024-09-17T09:27:28.093344517Z level=info msg="State cache has been initialized" states=0 duration=10.123325ms 2024-09-17T09:27:28.509 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=ticker t=2024-09-17T09:27:28.093391481Z level=info msg=starting first_tick=2024-09-17T09:27:30Z 2024-09-17T09:27:28.509 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=ngalert.multiorg.alertmanager t=2024-09-17T09:27:28.093406741Z level=info msg="starting MultiOrg Alertmanager" 2024-09-17T09:27:28.509 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=sqlstore.transactions t=2024-09-17T09:27:28.094082613Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T09:27:28.509 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a[51431]: logger=sqlstore.transactions t=2024-09-17T09:27:28.138086019Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-17T09:27:28.563 INFO:teuthology.orchestra.run.smithi148.stdout:77309411372 2024-09-17T09:27:28.564 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-17T09:27:29.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:28 smithi148 ceph-mon[25650]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:29.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:29.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:27:29.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:28 smithi148 ceph-mon[32020]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:29.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:29.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:27:29.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-mon[27402]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:29.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:29.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:27:29.876 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:27:29 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:27:29.933 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:30.086 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:29 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:30.087 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:29 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:30.259 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:29 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:30.360 INFO:teuthology.orchestra.run.smithi148.stdout:107374182440 2024-09-17T09:27:30.360 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-17T09:27:30.662 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:30 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ::ffff:172.21.15.157 - - [17/Sep/2024:09:27:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:27:30.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:30 smithi148 ceph-mon[32020]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:30.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:30 smithi148 ceph-mon[25650]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:31.140 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:31.164 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:30 smithi157 ceph-mon[27402]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:32.058 INFO:teuthology.orchestra.run.smithi148.stdout:128849018916 2024-09-17T09:27:32.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:32 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/1799400086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T09:27:32.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:32 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/1799400086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T09:27:32.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:32 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/1799400086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-17T09:27:32.824 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018913 got 128849018916 for osd.4 2024-09-17T09:27:32.824 DEBUG:teuthology.parallel:result is None 2024-09-17T09:27:33.362 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:33 smithi148 ceph-mon[32020]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:33.362 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:33 smithi148 ceph-mon[25650]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:33.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:33 smithi157 ceph-mon[27402]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:34.703 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:34.709 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:34.710 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:34.842 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:34 smithi148 ceph-mon[32020]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:34.842 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:34 smithi148 ceph-mon[25650]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:34.885 INFO:teuthology.orchestra.run.smithi148.stdout:231928234004 2024-09-17T09:27:35.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:34 smithi157 ceph-mon[27402]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:35.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:35 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3257344053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T09:27:35.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:35 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3257344053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T09:27:36.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:35 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3257344053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-17T09:27:36.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:36 smithi148 ceph-mon[25650]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:36.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:36 smithi148 ceph-mon[32020]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:37.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:36 smithi157 ceph-mon[27402]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:37.303 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:37 smithi148 bash[57437]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-17T09:27:37.667 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:37 smithi148 bash[57437]: Writing manifest to image destination 2024-09-17T09:27:38.167 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:37 smithi148 podman[57437]: 2024-09-17 09:27:37.817681188 +0000 UTC m=+3.108059043 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-17T09:27:38.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:38 smithi148 ceph-mon[25650]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:38.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:38 smithi148 ceph-mon[32020]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:39.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:38 smithi157 ceph-mon[27402]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:39.280 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:39.414 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 podman[57437]: 2024-09-17 09:27:39.274121812 +0000 UTC m=+4.564499729 container create e7344f84d46e53b3d20a73ec73cd61febdd92e7913a34f7ee69ad566ade81300 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:27:39.830 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 podman[57437]: 2024-09-17 09:27:39.720743678 +0000 UTC m=+5.011121533 container init e7344f84d46e53b3d20a73ec73cd61febdd92e7913a34f7ee69ad566ade81300 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:27:39.830 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 podman[57437]: 2024-09-17 09:27:39.723837178 +0000 UTC m=+5.014215033 container start e7344f84d46e53b3d20a73ec73cd61febdd92e7913a34f7ee69ad566ade81300 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:27:39.830 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.727Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T09:27:39.830 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.727Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T09:27:39.830 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T09:27:39.831 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T09:27:39.831 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T09:27:39.831 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T09:27:39.831 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T09:27:39.831 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T09:27:39.831 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T09:27:39.831 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T09:27:39.831 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T09:27:39.831 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T09:27:39.831 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T09:27:39.831 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T09:27:39.832 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.728Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T09:27:39.832 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T09:27:39.832 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T09:27:39.832 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T09:27:39.832 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T09:27:39.832 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T09:27:39.833 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T09:27:39.834 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T09:27:39.835 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T09:27:39.835 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T09:27:39.835 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T09:27:39.835 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T09:27:39.835 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T09:27:39.835 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T09:27:39.835 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T09:27:39.835 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T09:27:39.835 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a[57730]: ts=2024-09-17T09:27:39.729Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T09:27:39.835 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 bash[57437]: e7344f84d46e53b3d20a73ec73cd61febdd92e7913a34f7ee69ad566ade81300 2024-09-17T09:27:40.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:27:39 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:27:40.167 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:27:39 smithi148 systemd[1]: Started Ceph node-exporter.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:27:40.518 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ::ffff:172.21.15.157 - - [17/Sep/2024:09:27:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:27:40.853 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:40 smithi157 systemd[1]: Starting Ceph node-exporter.b for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:27:41.163 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:41.163 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:41.163 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:41.163 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:41.163 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[25650]: Deploying daemon node-exporter.b on smithi157 2024-09-17T09:27:41.163 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[25650]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:41.163 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:41.163 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:41.164 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:41.164 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:41.164 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[32020]: Deploying daemon node-exporter.b on smithi157 2024-09-17T09:27:41.164 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:40 smithi148 ceph-mon[32020]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:41.184 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:40 smithi157 bash[52355]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-17T09:27:41.184 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:41 smithi157 bash[52355]: Getting image source signatures 2024-09-17T09:27:41.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:40 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:41.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:40 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:41.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:40 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:41.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:40 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:41.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:40 smithi157 ceph-mon[27402]: Deploying daemon node-exporter.b on smithi157 2024-09-17T09:27:41.185 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:40 smithi157 ceph-mon[27402]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:41.523 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:41 smithi157 bash[52355]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-17T09:27:41.523 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:41 smithi157 bash[52355]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-17T09:27:41.523 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:41 smithi157 bash[52355]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-17T09:27:41.624 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234002 got 231928234004 for osd.7 2024-09-17T09:27:41.624 DEBUG:teuthology.parallel:result is None 2024-09-17T09:27:42.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:42 smithi157 ceph-mon[27402]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:42.774 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:42 smithi157 bash[52355]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-17T09:27:42.774 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:42 smithi157 bash[52355]: Writing manifest to image destination 2024-09-17T09:27:42.876 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:42 smithi148 ceph-mon[32020]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:42.876 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:42 smithi148 ceph-mon[25650]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:43.273 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:42 smithi157 podman[52355]: 2024-09-17 09:27:42.918449771 +0000 UTC m=+2.158452291 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-17T09:27:43.273 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 podman[52355]: 2024-09-17 09:27:43.025313208 +0000 UTC m=+2.265315720 container create 55850ab76da2f24467764b284aed2a1509f0fd22584381f4909276c33a516519 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:27:43.539 INFO:teuthology.orchestra.run.smithi148.stdout:163208757281 2024-09-17T09:27:43.774 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 podman[52355]: 2024-09-17 09:27:43.277441515 +0000 UTC m=+2.517444040 container init 55850ab76da2f24467764b284aed2a1509f0fd22584381f4909276c33a516519 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:27:43.774 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 podman[52355]: 2024-09-17 09:27:43.280417847 +0000 UTC m=+2.520420369 container start 55850ab76da2f24467764b284aed2a1509f0fd22584381f4909276c33a516519 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.284Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.284Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.284Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.284Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=arp 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-17T09:27:43.775 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-17T09:27:43.776 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-17T09:27:43.776 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-17T09:27:43.776 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-17T09:27:43.776 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=edac 2024-09-17T09:27:43.776 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-17T09:27:43.776 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-17T09:27:43.777 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-17T09:27:43.778 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-17T09:27:43.778 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-17T09:27:43.778 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=os 2024-09-17T09:27:43.778 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-17T09:27:43.778 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-17T09:27:43.778 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-17T09:27:43.778 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-17T09:27:43.778 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-17T09:27:43.778 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-17T09:27:43.778 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-17T09:27:43.778 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=stat 2024-09-17T09:27:43.779 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-17T09:27:43.779 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-17T09:27:43.779 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-17T09:27:43.779 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=time 2024-09-17T09:27:43.779 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-17T09:27:43.779 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=uname 2024-09-17T09:27:43.779 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-17T09:27:43.779 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-17T09:27:43.779 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.285Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-17T09:27:43.779 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.286Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-17T09:27:43.779 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b[52431]: ts=2024-09-17T09:27:43.286Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-17T09:27:43.780 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 bash[52355]: 55850ab76da2f24467764b284aed2a1509f0fd22584381f4909276c33a516519 2024-09-17T09:27:43.780 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:27:43 smithi157 systemd[1]: Started Ceph node-exporter.b for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:27:44.270 INFO:teuthology.orchestra.run.smithi148.stdout:197568495643 2024-09-17T09:27:44.551 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.551 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.551 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.551 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.551 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.551 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:27:44.551 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi148", "caps": []}]: dispatch 2024-09-17T09:27:44.551 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi157", "caps": []}]: dispatch 2024-09-17T09:27:44.551 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3336691958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T09:27:44.552 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.552 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:27:44.552 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/2518260810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T09:27:44.552 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.552 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.552 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.552 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.552 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.552 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:27:44.552 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi148", "caps": []}]: dispatch 2024-09-17T09:27:44.552 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi157", "caps": []}]: dispatch 2024-09-17T09:27:44.553 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3336691958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T09:27:44.553 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.553 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:27:44.553 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:44 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/2518260810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T09:27:44.566 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.566 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.566 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.566 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.566 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.566 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:27:44.566 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi148", "caps": []}]: dispatch 2024-09-17T09:27:44.566 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi157", "caps": []}]: dispatch 2024-09-17T09:27:44.566 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3336691958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-17T09:27:44.566 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:44.567 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:27:44.567 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:44 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/2518260810' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-17T09:27:45.158 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757277 got 163208757281 for osd.5 2024-09-17T09:27:45.158 DEBUG:teuthology.parallel:result is None 2024-09-17T09:27:45.667 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:45 smithi157 ceph-mon[27402]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:45.667 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:45 smithi157 ceph-mon[27402]: Reconfiguring daemon agent.smithi148 on smithi148 2024-09-17T09:27:45.667 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:45 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:45.856 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:45 smithi148 ceph-mon[25650]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:45.857 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:45 smithi148 ceph-mon[25650]: Reconfiguring daemon agent.smithi148 on smithi148 2024-09-17T09:27:45.857 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:45 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:45.857 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:45 smithi148 ceph-mon[32020]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:45.857 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:45 smithi148 ceph-mon[32020]: Reconfiguring daemon agent.smithi148 on smithi148 2024-09-17T09:27:45.857 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:45 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:45.878 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495638 got 197568495643 for osd.6 2024-09-17T09:27:45.878 DEBUG:teuthology.parallel:result is None 2024-09-17T09:27:46.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:46 smithi157 ceph-mon[27402]: Reconfiguring daemon agent.smithi157 on smithi157 2024-09-17T09:27:46.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:46 smithi148 ceph-mon[25650]: Reconfiguring daemon agent.smithi157 on smithi157 2024-09-17T09:27:46.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:46 smithi148 ceph-mon[32020]: Reconfiguring daemon agent.smithi157 on smithi157 2024-09-17T09:27:47.633 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:47 smithi148 ceph-mon[25650]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:47.633 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:47 smithi148 ceph-mon[32020]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:47.696 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:47 smithi157 ceph-mon[27402]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:48.178 INFO:teuthology.orchestra.run.smithi148.stdout:77309411376 2024-09-17T09:27:48.284 INFO:teuthology.orchestra.run.smithi148.stdout:55834574900 2024-09-17T09:27:48.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:48 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:48.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:48 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/1879982391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T09:27:48.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:48 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/214950799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T09:27:48.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:48 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:48.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:48 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/1879982391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T09:27:48.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:48 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/214950799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T09:27:48.766 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:48 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:27:48.767 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:48 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/1879982391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-17T09:27:48.767 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:48 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/214950799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-17T09:27:49.033 INFO:teuthology.orchestra.run.smithi148.stdout:34359738425 2024-09-17T09:27:49.363 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574896 got 55834574900 for osd.1 2024-09-17T09:27:49.364 DEBUG:teuthology.parallel:result is None 2024-09-17T09:27:49.618 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:27:49 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:27:49.618 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:49 smithi157 ceph-mon[27402]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:49.618 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:49 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/1904746133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T09:27:49.636 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:49 smithi148 ceph-mon[25650]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:49.636 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:49 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/1904746133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T09:27:49.636 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:49 smithi148 ceph-mon[32020]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:49.636 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:49 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/1904746133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-17T09:27:49.827 INFO:teuthology.orchestra.run.smithi148.stdout:107374182444 2024-09-17T09:27:50.506 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182440 got 107374182444 for osd.3 2024-09-17T09:27:50.506 DEBUG:teuthology.parallel:result is None 2024-09-17T09:27:50.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:50 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/3798792845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T09:27:50.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:27:50 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ::ffff:172.21.15.157 - - [17/Sep/2024:09:27:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:27:50.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:50 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/3798792845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T09:27:50.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:50 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/3798792845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-17T09:27:50.893 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411372 got 77309411376 for osd.2 2024-09-17T09:27:50.894 DEBUG:teuthology.parallel:result is None 2024-09-17T09:27:51.357 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738419 got 34359738425 for osd.0 2024-09-17T09:27:51.358 DEBUG:teuthology.parallel:result is None 2024-09-17T09:27:51.358 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-17T09:27:51.358 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T09:27:51.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:51 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:51.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:51 smithi148 ceph-mon[32020]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:51.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:51 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:51.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:51 smithi148 ceph-mon[25650]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:51.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:51 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:27:51.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:51 smithi157 ceph-mon[27402]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:52.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:52 smithi157 ceph-mon[27402]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:52.797 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:52 smithi148 ceph-mon[25650]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:52.798 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:52 smithi148 ceph-mon[32020]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:52.826 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:54.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:54 smithi148 ceph-mon[25650]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:54.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:54 smithi148 ceph-mon[32020]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:55.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:54 smithi157 ceph-mon[27402]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:56.328 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:27:56.330 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-09-17T09:27:56.912 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:56 smithi148 ceph-mon[32020]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:27:56.912 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:56 smithi148 ceph-mon[32020]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:56.912 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:56 smithi148 ceph-mon[25650]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:27:56.912 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:56 smithi148 ceph-mon[25650]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:57.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:56 smithi157 ceph-mon[27402]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:27:57.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:56 smithi157 ceph-mon[27402]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:27:57.270 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-17T09:27:54.537838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1054,"num_read_kb":897,"num_write":675,"num_write_kb":4025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":683,"ondisk_log_size":683,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":240480,"kb_used_data":7940,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":749687968,"statfs":{"total":767926730752,"available":767680479232,"internally_reserved":0,"allocated":8130560,"data_stored":4421598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002997"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474406+0000","last_change":"2024-09-17T09:26:21.248566+0000","last_active":"2024-09-17T09:26:46.474406+0000","last_peered":"2024-09-17T09:26:46.474406+0000","last_clean":"2024-09-17T09:26:46.474406+0000","last_became_active":"2024-09-17T09:26:21.248014+0000","last_became_peered":"2024-09-17T09:26:21.248014+0000","last_unstale":"2024-09-17T09:26:46.474406+0000","last_undegraded":"2024-09-17T09:26:46.474406+0000","last_fullsized":"2024-09-17T09:26:46.474406+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:20.315997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.272976+0000","last_change":"2024-09-17T09:26:14.723451+0000","last_active":"2024-09-17T09:26:47.272976+0000","last_peered":"2024-09-17T09:26:47.272976+0000","last_clean":"2024-09-17T09:26:47.272976+0000","last_became_active":"2024-09-17T09:26:14.723168+0000","last_became_peered":"2024-09-17T09:26:14.723168+0000","last_unstale":"2024-09-17T09:26:47.272976+0000","last_undegraded":"2024-09-17T09:26:47.272976+0000","last_fullsized":"2024-09-17T09:26:47.272976+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:09.679630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474927+0000","last_change":"2024-09-17T09:26:16.812395+0000","last_active":"2024-09-17T09:26:46.474927+0000","last_peered":"2024-09-17T09:26:46.474927+0000","last_clean":"2024-09-17T09:26:46.474927+0000","last_became_active":"2024-09-17T09:26:16.812016+0000","last_became_peered":"2024-09-17T09:26:16.812016+0000","last_unstale":"2024-09-17T09:26:46.474927+0000","last_undegraded":"2024-09-17T09:26:46.474927+0000","last_fullsized":"2024-09-17T09:26:46.474927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:16.338194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.870767+0000","last_change":"2024-09-17T09:26:18.761963+0000","last_active":"2024-09-17T09:26:46.870767+0000","last_peered":"2024-09-17T09:26:46.870767+0000","last_clean":"2024-09-17T09:26:46.870767+0000","last_became_active":"2024-09-17T09:26:18.761670+0000","last_became_peered":"2024-09-17T09:26:18.761670+0000","last_unstale":"2024-09-17T09:26:46.870767+0000","last_undegraded":"2024-09-17T09:26:46.870767+0000","last_fullsized":"2024-09-17T09:26:46.870767+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:33.290068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474997+0000","last_change":"2024-09-17T09:26:14.726819+0000","last_active":"2024-09-17T09:26:46.474997+0000","last_peered":"2024-09-17T09:26:46.474997+0000","last_clean":"2024-09-17T09:26:46.474997+0000","last_became_active":"2024-09-17T09:26:14.726551+0000","last_became_peered":"2024-09-17T09:26:14.726551+0000","last_unstale":"2024-09-17T09:26:46.474997+0000","last_undegraded":"2024-09-17T09:26:46.474997+0000","last_fullsized":"2024-09-17T09:26:46.474997+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:36:14.494642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273979+0000","last_change":"2024-09-17T09:26:16.812463+0000","last_active":"2024-09-17T09:26:47.273979+0000","last_peered":"2024-09-17T09:26:47.273979+0000","last_clean":"2024-09-17T09:26:47.273979+0000","last_became_active":"2024-09-17T09:26:16.812187+0000","last_became_peered":"2024-09-17T09:26:16.812187+0000","last_unstale":"2024-09-17T09:26:47.273979+0000","last_undegraded":"2024-09-17T09:26:47.273979+0000","last_fullsized":"2024-09-17T09:26:47.273979+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:51.660034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756978+0000","last_change":"2024-09-17T09:26:18.761794+0000","last_active":"2024-09-17T09:26:46.756978+0000","last_peered":"2024-09-17T09:26:46.756978+0000","last_clean":"2024-09-17T09:26:46.756978+0000","last_became_active":"2024-09-17T09:26:18.761442+0000","last_became_peered":"2024-09-17T09:26:18.761442+0000","last_unstale":"2024-09-17T09:26:46.756978+0000","last_undegraded":"2024-09-17T09:26:46.756978+0000","last_fullsized":"2024-09-17T09:26:46.756978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:19.724993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871049+0000","last_change":"2024-09-17T09:26:20.769283+0000","last_active":"2024-09-17T09:26:46.871049+0000","last_peered":"2024-09-17T09:26:46.871049+0000","last_clean":"2024-09-17T09:26:46.871049+0000","last_became_active":"2024-09-17T09:26:20.768995+0000","last_became_peered":"2024-09-17T09:26:20.768995+0000","last_unstale":"2024-09-17T09:26:46.871049+0000","last_undegraded":"2024-09-17T09:26:46.871049+0000","last_fullsized":"2024-09-17T09:26:46.871049+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:10:33.969739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474394+0000","last_change":"2024-09-17T09:26:14.732633+0000","last_active":"2024-09-17T09:26:46.474394+0000","last_peered":"2024-09-17T09:26:46.474394+0000","last_clean":"2024-09-17T09:26:46.474394+0000","last_became_active":"2024-09-17T09:26:14.732292+0000","last_became_peered":"2024-09-17T09:26:14.732292+0000","last_unstale":"2024-09-17T09:26:46.474394+0000","last_undegraded":"2024-09-17T09:26:46.474394+0000","last_fullsized":"2024-09-17T09:26:46.474394+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:48.218891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476075+0000","last_change":"2024-09-17T09:26:16.745776+0000","last_active":"2024-09-17T09:26:46.476075+0000","last_peered":"2024-09-17T09:26:46.476075+0000","last_clean":"2024-09-17T09:26:46.476075+0000","last_became_active":"2024-09-17T09:26:16.745524+0000","last_became_peered":"2024-09-17T09:26:16.745524+0000","last_unstale":"2024-09-17T09:26:46.476075+0000","last_undegraded":"2024-09-17T09:26:46.476075+0000","last_fullsized":"2024-09-17T09:26:46.476075+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:54:01.177151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473646+0000","last_change":"2024-09-17T09:26:18.762720+0000","last_active":"2024-09-17T09:26:46.473646+0000","last_peered":"2024-09-17T09:26:46.473646+0000","last_clean":"2024-09-17T09:26:46.473646+0000","last_became_active":"2024-09-17T09:26:18.762400+0000","last_became_peered":"2024-09-17T09:26:18.762400+0000","last_unstale":"2024-09-17T09:26:46.473646+0000","last_undegraded":"2024-09-17T09:26:46.473646+0000","last_fullsized":"2024-09-17T09:26:46.473646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:39.153980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476154+0000","last_change":"2024-09-17T09:26:20.779004+0000","last_active":"2024-09-17T09:26:46.476154+0000","last_peered":"2024-09-17T09:26:46.476154+0000","last_clean":"2024-09-17T09:26:46.476154+0000","last_became_active":"2024-09-17T09:26:20.778421+0000","last_became_peered":"2024-09-17T09:26:20.778421+0000","last_unstale":"2024-09-17T09:26:46.476154+0000","last_undegraded":"2024-09-17T09:26:46.476154+0000","last_fullsized":"2024-09-17T09:26:46.476154+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:26.139869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474163+0000","last_change":"2024-09-17T09:26:14.728651+0000","last_active":"2024-09-17T09:26:46.474163+0000","last_peered":"2024-09-17T09:26:46.474163+0000","last_clean":"2024-09-17T09:26:46.474163+0000","last_became_active":"2024-09-17T09:26:14.728371+0000","last_became_peered":"2024-09-17T09:26:14.728371+0000","last_unstale":"2024-09-17T09:26:46.474163+0000","last_undegraded":"2024-09-17T09:26:46.474163+0000","last_fullsized":"2024-09-17T09:26:46.474163+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:40.009267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"67'12","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476313+0000","last_change":"2024-09-17T09:26:16.743382+0000","last_active":"2024-09-17T09:26:46.476313+0000","last_peered":"2024-09-17T09:26:46.476313+0000","last_clean":"2024-09-17T09:26:46.476313+0000","last_became_active":"2024-09-17T09:26:16.743123+0000","last_became_peered":"2024-09-17T09:26:16.743123+0000","last_unstale":"2024-09-17T09:26:46.476313+0000","last_undegraded":"2024-09-17T09:26:46.476313+0000","last_fullsized":"2024-09-17T09:26:46.476313+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:36:30.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.475789+0000","last_change":"2024-09-17T09:26:18.764655+0000","last_active":"2024-09-17T09:26:46.475789+0000","last_peered":"2024-09-17T09:26:46.475789+0000","last_clean":"2024-09-17T09:26:46.475789+0000","last_became_active":"2024-09-17T09:26:18.764365+0000","last_became_peered":"2024-09-17T09:26:18.764365+0000","last_unstale":"2024-09-17T09:26:46.475789+0000","last_undegraded":"2024-09-17T09:26:46.475789+0000","last_fullsized":"2024-09-17T09:26:46.475789+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:41:22.891498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273246+0000","last_change":"2024-09-17T09:26:20.774839+0000","last_active":"2024-09-17T09:26:47.273246+0000","last_peered":"2024-09-17T09:26:47.273246+0000","last_clean":"2024-09-17T09:26:47.273246+0000","last_became_active":"2024-09-17T09:26:20.774527+0000","last_became_peered":"2024-09-17T09:26:20.774527+0000","last_unstale":"2024-09-17T09:26:47.273246+0000","last_undegraded":"2024-09-17T09:26:47.273246+0000","last_fullsized":"2024-09-17T09:26:47.273246+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:27.642478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"67'19","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223854+0000","last_change":"2024-09-17T09:26:16.745913+0000","last_active":"2024-09-17T09:26:47.223854+0000","last_peered":"2024-09-17T09:26:47.223854+0000","last_clean":"2024-09-17T09:26:47.223854+0000","last_became_active":"2024-09-17T09:26:16.745655+0000","last_became_peered":"2024-09-17T09:26:16.745655+0000","last_unstale":"2024-09-17T09:26:47.223854+0000","last_undegraded":"2024-09-17T09:26:47.223854+0000","last_fullsized":"2024-09-17T09:26:47.223854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:33.404335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474155+0000","last_change":"2024-09-17T09:26:14.728051+0000","last_active":"2024-09-17T09:26:46.474155+0000","last_peered":"2024-09-17T09:26:46.474155+0000","last_clean":"2024-09-17T09:26:46.474155+0000","last_became_active":"2024-09-17T09:26:14.727749+0000","last_became_peered":"2024-09-17T09:26:14.727749+0000","last_unstale":"2024-09-17T09:26:46.474155+0000","last_undegraded":"2024-09-17T09:26:46.474155+0000","last_fullsized":"2024-09-17T09:26:46.474155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:52:08.725435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757283+0000","last_change":"2024-09-17T09:26:18.762298+0000","last_active":"2024-09-17T09:26:46.757283+0000","last_peered":"2024-09-17T09:26:46.757283+0000","last_clean":"2024-09-17T09:26:46.757283+0000","last_became_active":"2024-09-17T09:26:18.761985+0000","last_became_peered":"2024-09-17T09:26:18.761985+0000","last_unstale":"2024-09-17T09:26:46.757283+0000","last_undegraded":"2024-09-17T09:26:46.757283+0000","last_fullsized":"2024-09-17T09:26:46.757283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:45.470662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027869+0000","last_change":"2024-09-17T09:26:20.777657+0000","last_active":"2024-09-17T09:26:47.027869+0000","last_peered":"2024-09-17T09:26:47.027869+0000","last_clean":"2024-09-17T09:26:47.027869+0000","last_became_active":"2024-09-17T09:26:20.777315+0000","last_became_peered":"2024-09-17T09:26:20.777315+0000","last_unstale":"2024-09-17T09:26:47.027869+0000","last_undegraded":"2024-09-17T09:26:47.027869+0000","last_fullsized":"2024-09-17T09:26:47.027869+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:41.814566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474533+0000","last_change":"2024-09-17T09:26:16.745545+0000","last_active":"2024-09-17T09:26:46.474533+0000","last_peered":"2024-09-17T09:26:46.474533+0000","last_clean":"2024-09-17T09:26:46.474533+0000","last_became_active":"2024-09-17T09:26:16.745284+0000","last_became_peered":"2024-09-17T09:26:16.745284+0000","last_unstale":"2024-09-17T09:26:46.474533+0000","last_undegraded":"2024-09-17T09:26:46.474533+0000","last_fullsized":"2024-09-17T09:26:46.474533+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:55:25.499228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756620+0000","last_change":"2024-09-17T09:26:14.724081+0000","last_active":"2024-09-17T09:26:46.756620+0000","last_peered":"2024-09-17T09:26:46.756620+0000","last_clean":"2024-09-17T09:26:46.756620+0000","last_became_active":"2024-09-17T09:26:14.723767+0000","last_became_peered":"2024-09-17T09:26:14.723767+0000","last_unstale":"2024-09-17T09:26:46.756620+0000","last_undegraded":"2024-09-17T09:26:46.756620+0000","last_fullsized":"2024-09-17T09:26:46.756620+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:16.751196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:47.947384+0000","last_change":"2024-09-17T09:26:18.762481+0000","last_active":"2024-09-17T09:27:47.947384+0000","last_peered":"2024-09-17T09:27:47.947384+0000","last_clean":"2024-09-17T09:27:47.947384+0000","last_became_active":"2024-09-17T09:26:18.762206+0000","last_became_peered":"2024-09-17T09:26:18.762206+0000","last_unstale":"2024-09-17T09:27:47.947384+0000","last_undegraded":"2024-09-17T09:27:47.947384+0000","last_fullsized":"2024-09-17T09:27:47.947384+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:48.844460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871396+0000","last_change":"2024-09-17T09:26:20.781202+0000","last_active":"2024-09-17T09:26:46.871396+0000","last_peered":"2024-09-17T09:26:46.871396+0000","last_clean":"2024-09-17T09:26:46.871396+0000","last_became_active":"2024-09-17T09:26:20.780480+0000","last_became_peered":"2024-09-17T09:26:20.780480+0000","last_unstale":"2024-09-17T09:26:46.871396+0000","last_undegraded":"2024-09-17T09:26:46.871396+0000","last_fullsized":"2024-09-17T09:26:46.871396+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:38.663263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474076+0000","last_change":"2024-09-17T09:26:16.747931+0000","last_active":"2024-09-17T09:26:46.474076+0000","last_peered":"2024-09-17T09:26:46.474076+0000","last_clean":"2024-09-17T09:26:46.474076+0000","last_became_active":"2024-09-17T09:26:16.747579+0000","last_became_peered":"2024-09-17T09:26:16.747579+0000","last_unstale":"2024-09-17T09:26:46.474076+0000","last_undegraded":"2024-09-17T09:26:46.474076+0000","last_fullsized":"2024-09-17T09:26:46.474076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:56:43.122018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756554+0000","last_change":"2024-09-17T09:26:14.725459+0000","last_active":"2024-09-17T09:26:46.756554+0000","last_peered":"2024-09-17T09:26:46.756554+0000","last_clean":"2024-09-17T09:26:46.756554+0000","last_became_active":"2024-09-17T09:26:14.725204+0000","last_became_peered":"2024-09-17T09:26:14.725204+0000","last_unstale":"2024-09-17T09:26:46.756554+0000","last_undegraded":"2024-09-17T09:26:46.756554+0000","last_fullsized":"2024-09-17T09:26:46.756554+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:45.348231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:47.947477+0000","last_change":"2024-09-17T09:26:18.764740+0000","last_active":"2024-09-17T09:27:47.947477+0000","last_peered":"2024-09-17T09:27:47.947477+0000","last_clean":"2024-09-17T09:27:47.947477+0000","last_became_active":"2024-09-17T09:26:18.764479+0000","last_became_peered":"2024-09-17T09:26:18.764479+0000","last_unstale":"2024-09-17T09:27:47.947477+0000","last_undegraded":"2024-09-17T09:27:47.947477+0000","last_fullsized":"2024-09-17T09:27:47.947477+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:45.303237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476087+0000","last_change":"2024-09-17T09:26:20.775266+0000","last_active":"2024-09-17T09:26:46.476087+0000","last_peered":"2024-09-17T09:26:46.476087+0000","last_clean":"2024-09-17T09:26:46.476087+0000","last_became_active":"2024-09-17T09:26:20.774891+0000","last_became_peered":"2024-09-17T09:26:20.774891+0000","last_unstale":"2024-09-17T09:26:46.476087+0000","last_undegraded":"2024-09-17T09:26:46.476087+0000","last_fullsized":"2024-09-17T09:26:46.476087+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:25:05.846709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"67'12","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871191+0000","last_change":"2024-09-17T09:26:16.813501+0000","last_active":"2024-09-17T09:26:46.871191+0000","last_peered":"2024-09-17T09:26:46.871191+0000","last_clean":"2024-09-17T09:26:46.871191+0000","last_became_active":"2024-09-17T09:26:16.813196+0000","last_became_peered":"2024-09-17T09:26:16.813196+0000","last_unstale":"2024-09-17T09:26:46.871191+0000","last_undegraded":"2024-09-17T09:26:46.871191+0000","last_fullsized":"2024-09-17T09:26:46.871191+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:39.868980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473517+0000","last_change":"2024-09-17T09:26:14.728109+0000","last_active":"2024-09-17T09:26:46.473517+0000","last_peered":"2024-09-17T09:26:46.473517+0000","last_clean":"2024-09-17T09:26:46.473517+0000","last_became_active":"2024-09-17T09:26:14.727819+0000","last_became_peered":"2024-09-17T09:26:14.727819+0000","last_unstale":"2024-09-17T09:26:46.473517+0000","last_undegraded":"2024-09-17T09:26:46.473517+0000","last_fullsized":"2024-09-17T09:26:46.473517+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:02.267972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476360+0000","last_change":"2024-09-17T09:26:18.764783+0000","last_active":"2024-09-17T09:26:46.476360+0000","last_peered":"2024-09-17T09:26:46.476360+0000","last_clean":"2024-09-17T09:26:46.476360+0000","last_became_active":"2024-09-17T09:26:18.763820+0000","last_became_peered":"2024-09-17T09:26:18.763820+0000","last_unstale":"2024-09-17T09:26:46.476360+0000","last_undegraded":"2024-09-17T09:26:46.476360+0000","last_fullsized":"2024-09-17T09:26:46.476360+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:36:24.449240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474508+0000","last_change":"2024-09-17T09:26:21.247439+0000","last_active":"2024-09-17T09:26:46.474508+0000","last_peered":"2024-09-17T09:26:46.474508+0000","last_clean":"2024-09-17T09:26:46.474508+0000","last_became_active":"2024-09-17T09:26:21.247088+0000","last_became_peered":"2024-09-17T09:26:21.247088+0000","last_unstale":"2024-09-17T09:26:46.474508+0000","last_undegraded":"2024-09-17T09:26:46.474508+0000","last_fullsized":"2024-09-17T09:26:46.474508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:06.886592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"67'12","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273914+0000","last_change":"2024-09-17T09:26:16.743884+0000","last_active":"2024-09-17T09:26:47.273914+0000","last_peered":"2024-09-17T09:26:47.273914+0000","last_clean":"2024-09-17T09:26:47.273914+0000","last_became_active":"2024-09-17T09:26:16.743580+0000","last_became_peered":"2024-09-17T09:26:16.743580+0000","last_unstale":"2024-09-17T09:26:47.273914+0000","last_undegraded":"2024-09-17T09:26:47.273914+0000","last_fullsized":"2024-09-17T09:26:47.273914+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:00.691635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223329+0000","last_change":"2024-09-17T09:26:14.724391+0000","last_active":"2024-09-17T09:26:47.223329+0000","last_peered":"2024-09-17T09:26:47.223329+0000","last_clean":"2024-09-17T09:26:47.223329+0000","last_became_active":"2024-09-17T09:26:14.724196+0000","last_became_peered":"2024-09-17T09:26:14.724196+0000","last_unstale":"2024-09-17T09:26:47.223329+0000","last_undegraded":"2024-09-17T09:26:47.223329+0000","last_fullsized":"2024-09-17T09:26:47.223329+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:01.616900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'1","reported_seq":25,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871049+0000","last_change":"2024-09-17T09:26:20.760952+0000","last_active":"2024-09-17T09:26:46.871049+0000","last_peered":"2024-09-17T09:26:46.871049+0000","last_clean":"2024-09-17T09:26:46.871049+0000","last_became_active":"2024-09-17T09:26:16.745279+0000","last_became_peered":"2024-09-17T09:26:16.745279+0000","last_unstale":"2024-09-17T09:26:46.871049+0000","last_undegraded":"2024-09-17T09:26:46.871049+0000","last_fullsized":"2024-09-17T09:26:46.871049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:49:29.689044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000380288,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:47.947375+0000","last_change":"2024-09-17T09:26:18.760293+0000","last_active":"2024-09-17T09:27:47.947375+0000","last_peered":"2024-09-17T09:27:47.947375+0000","last_clean":"2024-09-17T09:27:47.947375+0000","last_became_active":"2024-09-17T09:26:18.759860+0000","last_became_peered":"2024-09-17T09:26:18.759860+0000","last_unstale":"2024-09-17T09:27:47.947375+0000","last_undegraded":"2024-09-17T09:27:47.947375+0000","last_fullsized":"2024-09-17T09:27:47.947375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:22.903002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473615+0000","last_change":"2024-09-17T09:26:21.250808+0000","last_active":"2024-09-17T09:26:46.473615+0000","last_peered":"2024-09-17T09:26:46.473615+0000","last_clean":"2024-09-17T09:26:46.473615+0000","last_became_active":"2024-09-17T09:26:21.250520+0000","last_became_peered":"2024-09-17T09:26:21.250520+0000","last_unstale":"2024-09-17T09:26:46.473615+0000","last_undegraded":"2024-09-17T09:26:46.473615+0000","last_fullsized":"2024-09-17T09:26:46.473615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:02:00.601554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"67'13","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.475038+0000","last_change":"2024-09-17T09:26:16.748071+0000","last_active":"2024-09-17T09:26:46.475038+0000","last_peered":"2024-09-17T09:26:46.475038+0000","last_clean":"2024-09-17T09:26:46.475038+0000","last_became_active":"2024-09-17T09:26:16.747767+0000","last_became_peered":"2024-09-17T09:26:16.747767+0000","last_unstale":"2024-09-17T09:26:46.475038+0000","last_undegraded":"2024-09-17T09:26:46.475038+0000","last_fullsized":"2024-09-17T09:26:46.475038+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:17:33.397506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757657+0000","last_change":"2024-09-17T09:26:14.721028+0000","last_active":"2024-09-17T09:26:46.757657+0000","last_peered":"2024-09-17T09:26:46.757657+0000","last_clean":"2024-09-17T09:26:46.757657+0000","last_became_active":"2024-09-17T09:26:14.720662+0000","last_became_peered":"2024-09-17T09:26:14.720662+0000","last_unstale":"2024-09-17T09:26:46.757657+0000","last_undegraded":"2024-09-17T09:26:46.757657+0000","last_fullsized":"2024-09-17T09:26:46.757657+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:53.527988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"68'5","reported_seq":119,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:53.279754+0000","last_change":"2024-09-17T09:26:20.775567+0000","last_active":"2024-09-17T09:27:53.279754+0000","last_peered":"2024-09-17T09:27:53.279754+0000","last_clean":"2024-09-17T09:27:53.279754+0000","last_became_active":"2024-09-17T09:26:16.747710+0000","last_became_peered":"2024-09-17T09:26:16.747710+0000","last_unstale":"2024-09-17T09:27:53.279754+0000","last_undegraded":"2024-09-17T09:27:53.279754+0000","last_fullsized":"2024-09-17T09:27:53.279754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:46.061292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028468,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.870868+0000","last_change":"2024-09-17T09:26:18.760591+0000","last_active":"2024-09-17T09:26:46.870868+0000","last_peered":"2024-09-17T09:26:46.870868+0000","last_clean":"2024-09-17T09:26:46.870868+0000","last_became_active":"2024-09-17T09:26:18.760318+0000","last_became_peered":"2024-09-17T09:26:18.760318+0000","last_unstale":"2024-09-17T09:26:46.870868+0000","last_undegraded":"2024-09-17T09:26:46.870868+0000","last_fullsized":"2024-09-17T09:26:46.870868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:46:37.291405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.870948+0000","last_change":"2024-09-17T09:26:20.777409+0000","last_active":"2024-09-17T09:26:46.870948+0000","last_peered":"2024-09-17T09:26:46.870948+0000","last_clean":"2024-09-17T09:26:46.870948+0000","last_became_active":"2024-09-17T09:26:20.777098+0000","last_became_peered":"2024-09-17T09:26:20.777098+0000","last_unstale":"2024-09-17T09:26:46.870948+0000","last_undegraded":"2024-09-17T09:26:46.870948+0000","last_fullsized":"2024-09-17T09:26:46.870948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:12.837968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"67'30","reported_seq":90,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:47.947413+0000","last_change":"2024-09-17T09:26:16.812376+0000","last_active":"2024-09-17T09:27:47.947413+0000","last_peered":"2024-09-17T09:27:47.947413+0000","last_clean":"2024-09-17T09:27:47.947413+0000","last_became_active":"2024-09-17T09:26:16.812057+0000","last_became_peered":"2024-09-17T09:26:16.812057+0000","last_unstale":"2024-09-17T09:27:47.947413+0000","last_undegraded":"2024-09-17T09:27:47.947413+0000","last_fullsized":"2024-09-17T09:27:47.947413+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:50.650693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474422+0000","last_change":"2024-09-17T09:26:14.732665+0000","last_active":"2024-09-17T09:26:46.474422+0000","last_peered":"2024-09-17T09:26:46.474422+0000","last_clean":"2024-09-17T09:26:46.474422+0000","last_became_active":"2024-09-17T09:26:14.732389+0000","last_became_peered":"2024-09-17T09:26:14.732389+0000","last_unstale":"2024-09-17T09:26:46.474422+0000","last_undegraded":"2024-09-17T09:26:46.474422+0000","last_fullsized":"2024-09-17T09:26:46.474422+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:11:46.814321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223749+0000","last_change":"2024-09-17T09:26:18.767936+0000","last_active":"2024-09-17T09:26:47.223749+0000","last_peered":"2024-09-17T09:26:47.223749+0000","last_clean":"2024-09-17T09:26:47.223749+0000","last_became_active":"2024-09-17T09:26:18.767650+0000","last_became_peered":"2024-09-17T09:26:18.767650+0000","last_unstale":"2024-09-17T09:26:47.223749+0000","last_undegraded":"2024-09-17T09:26:47.223749+0000","last_fullsized":"2024-09-17T09:26:47.223749+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:42.177315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756914+0000","last_change":"2024-09-17T09:26:21.248934+0000","last_active":"2024-09-17T09:26:46.756914+0000","last_peered":"2024-09-17T09:26:46.756914+0000","last_clean":"2024-09-17T09:26:46.756914+0000","last_became_active":"2024-09-17T09:26:21.248575+0000","last_became_peered":"2024-09-17T09:26:21.248575+0000","last_unstale":"2024-09-17T09:26:46.756914+0000","last_undegraded":"2024-09-17T09:26:46.756914+0000","last_fullsized":"2024-09-17T09:26:46.756914+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:39.167812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"67'14","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476301+0000","last_change":"2024-09-17T09:26:16.812435+0000","last_active":"2024-09-17T09:26:46.476301+0000","last_peered":"2024-09-17T09:26:46.476301+0000","last_clean":"2024-09-17T09:26:46.476301+0000","last_became_active":"2024-09-17T09:26:16.812159+0000","last_became_peered":"2024-09-17T09:26:16.812159+0000","last_unstale":"2024-09-17T09:26:46.476301+0000","last_undegraded":"2024-09-17T09:26:46.476301+0000","last_fullsized":"2024-09-17T09:26:46.476301+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:34:57.633416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756610+0000","last_change":"2024-09-17T09:26:14.728946+0000","last_active":"2024-09-17T09:26:46.756610+0000","last_peered":"2024-09-17T09:26:46.756610+0000","last_clean":"2024-09-17T09:26:46.756610+0000","last_became_active":"2024-09-17T09:26:14.728645+0000","last_became_peered":"2024-09-17T09:26:14.728645+0000","last_unstale":"2024-09-17T09:26:46.756610+0000","last_undegraded":"2024-09-17T09:26:46.756610+0000","last_fullsized":"2024-09-17T09:26:46.756610+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:53.731440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"66'2","reported_seq":26,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757295+0000","last_change":"2024-09-17T09:26:20.767626+0000","last_active":"2024-09-17T09:26:46.757295+0000","last_peered":"2024-09-17T09:26:46.757295+0000","last_clean":"2024-09-17T09:26:46.757295+0000","last_became_active":"2024-09-17T09:26:16.747334+0000","last_became_peered":"2024-09-17T09:26:16.747334+0000","last_unstale":"2024-09-17T09:26:46.757295+0000","last_undegraded":"2024-09-17T09:26:46.757295+0000","last_fullsized":"2024-09-17T09:26:46.757295+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.735859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013679499999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:47.947379+0000","last_change":"2024-09-17T09:26:18.765998+0000","last_active":"2024-09-17T09:27:47.947379+0000","last_peered":"2024-09-17T09:27:47.947379+0000","last_clean":"2024-09-17T09:27:47.947379+0000","last_became_active":"2024-09-17T09:26:18.765705+0000","last_became_peered":"2024-09-17T09:26:18.765705+0000","last_unstale":"2024-09-17T09:27:47.947379+0000","last_undegraded":"2024-09-17T09:27:47.947379+0000","last_fullsized":"2024-09-17T09:27:47.947379+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:57:21.622471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273149+0000","last_change":"2024-09-17T09:26:20.780175+0000","last_active":"2024-09-17T09:26:47.273149+0000","last_peered":"2024-09-17T09:26:47.273149+0000","last_clean":"2024-09-17T09:26:47.273149+0000","last_became_active":"2024-09-17T09:26:20.779862+0000","last_became_peered":"2024-09-17T09:26:20.779862+0000","last_unstale":"2024-09-17T09:26:47.273149+0000","last_undegraded":"2024-09-17T09:26:47.273149+0000","last_fullsized":"2024-09-17T09:26:47.273149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:20:42.666024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"67'19","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871355+0000","last_change":"2024-09-17T09:26:16.744687+0000","last_active":"2024-09-17T09:26:46.871355+0000","last_peered":"2024-09-17T09:26:46.871355+0000","last_clean":"2024-09-17T09:26:46.871355+0000","last_became_active":"2024-09-17T09:26:16.744078+0000","last_became_peered":"2024-09-17T09:26:16.744078+0000","last_unstale":"2024-09-17T09:26:46.871355+0000","last_undegraded":"2024-09-17T09:26:46.871355+0000","last_fullsized":"2024-09-17T09:26:46.871355+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:11.409204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'1","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476083+0000","last_change":"2024-09-17T09:26:14.724555+0000","last_active":"2024-09-17T09:26:46.476083+0000","last_peered":"2024-09-17T09:26:46.476083+0000","last_clean":"2024-09-17T09:26:46.476083+0000","last_became_active":"2024-09-17T09:26:14.724253+0000","last_became_peered":"2024-09-17T09:26:14.724253+0000","last_unstale":"2024-09-17T09:26:46.476083+0000","last_undegraded":"2024-09-17T09:26:46.476083+0000","last_fullsized":"2024-09-17T09:26:46.476083+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:04:44.010016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273749+0000","last_change":"2024-09-17T09:26:18.765039+0000","last_active":"2024-09-17T09:26:47.273749+0000","last_peered":"2024-09-17T09:26:47.273749+0000","last_clean":"2024-09-17T09:26:47.273749+0000","last_became_active":"2024-09-17T09:26:18.764779+0000","last_became_peered":"2024-09-17T09:26:18.764779+0000","last_unstale":"2024-09-17T09:26:47.273749+0000","last_undegraded":"2024-09-17T09:26:47.273749+0000","last_fullsized":"2024-09-17T09:26:47.273749+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:17.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474808+0000","last_change":"2024-09-17T09:26:20.780934+0000","last_active":"2024-09-17T09:26:46.474808+0000","last_peered":"2024-09-17T09:26:46.474808+0000","last_clean":"2024-09-17T09:26:46.474808+0000","last_became_active":"2024-09-17T09:26:20.780602+0000","last_became_peered":"2024-09-17T09:26:20.780602+0000","last_unstale":"2024-09-17T09:26:46.474808+0000","last_undegraded":"2024-09-17T09:26:46.474808+0000","last_fullsized":"2024-09-17T09:26:46.474808+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:33.807274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"67'18","reported_seq":51,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757172+0000","last_change":"2024-09-17T09:26:16.813661+0000","last_active":"2024-09-17T09:26:46.757172+0000","last_peered":"2024-09-17T09:26:46.757172+0000","last_clean":"2024-09-17T09:26:46.757172+0000","last_became_active":"2024-09-17T09:26:16.813407+0000","last_became_peered":"2024-09-17T09:26:16.813407+0000","last_unstale":"2024-09-17T09:26:46.757172+0000","last_undegraded":"2024-09-17T09:26:46.757172+0000","last_fullsized":"2024-09-17T09:26:46.757172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:44.012466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027047+0000","last_change":"2024-09-17T09:26:14.731301+0000","last_active":"2024-09-17T09:26:47.027047+0000","last_peered":"2024-09-17T09:26:47.027047+0000","last_clean":"2024-09-17T09:26:47.027047+0000","last_became_active":"2024-09-17T09:26:14.731022+0000","last_became_peered":"2024-09-17T09:26:14.731022+0000","last_unstale":"2024-09-17T09:26:47.027047+0000","last_undegraded":"2024-09-17T09:26:47.027047+0000","last_fullsized":"2024-09-17T09:26:47.027047+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:42.821585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027807+0000","last_change":"2024-09-17T09:26:18.762625+0000","last_active":"2024-09-17T09:26:47.027807+0000","last_peered":"2024-09-17T09:26:47.027807+0000","last_clean":"2024-09-17T09:26:47.027807+0000","last_became_active":"2024-09-17T09:26:18.762319+0000","last_became_peered":"2024-09-17T09:26:18.762319+0000","last_unstale":"2024-09-17T09:26:47.027807+0000","last_undegraded":"2024-09-17T09:26:47.027807+0000","last_fullsized":"2024-09-17T09:26:47.027807+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:45:30.058597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474056+0000","last_change":"2024-09-17T09:26:21.247196+0000","last_active":"2024-09-17T09:26:46.474056+0000","last_peered":"2024-09-17T09:26:46.474056+0000","last_clean":"2024-09-17T09:26:46.474056+0000","last_became_active":"2024-09-17T09:26:21.246857+0000","last_became_peered":"2024-09-17T09:26:21.246857+0000","last_unstale":"2024-09-17T09:26:46.474056+0000","last_undegraded":"2024-09-17T09:26:46.474056+0000","last_fullsized":"2024-09-17T09:26:46.474056+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:39.740778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"67'14","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273289+0000","last_change":"2024-09-17T09:26:16.746320+0000","last_active":"2024-09-17T09:26:47.273289+0000","last_peered":"2024-09-17T09:26:47.273289+0000","last_clean":"2024-09-17T09:26:47.273289+0000","last_became_active":"2024-09-17T09:26:16.746043+0000","last_became_peered":"2024-09-17T09:26:16.746043+0000","last_unstale":"2024-09-17T09:26:47.273289+0000","last_undegraded":"2024-09-17T09:26:47.273289+0000","last_fullsized":"2024-09-17T09:26:47.273289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:00.208644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473535+0000","last_change":"2024-09-17T09:26:14.728452+0000","last_active":"2024-09-17T09:26:46.473535+0000","last_peered":"2024-09-17T09:26:46.473535+0000","last_clean":"2024-09-17T09:26:46.473535+0000","last_became_active":"2024-09-17T09:26:14.728170+0000","last_became_peered":"2024-09-17T09:26:14.728170+0000","last_unstale":"2024-09-17T09:26:46.473535+0000","last_undegraded":"2024-09-17T09:26:46.473535+0000","last_fullsized":"2024-09-17T09:26:46.473535+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:59.581480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476496+0000","last_change":"2024-09-17T09:26:18.764967+0000","last_active":"2024-09-17T09:26:46.476496+0000","last_peered":"2024-09-17T09:26:46.476496+0000","last_clean":"2024-09-17T09:26:46.476496+0000","last_became_active":"2024-09-17T09:26:18.764527+0000","last_became_peered":"2024-09-17T09:26:18.764527+0000","last_unstale":"2024-09-17T09:26:46.476496+0000","last_undegraded":"2024-09-17T09:26:46.476496+0000","last_fullsized":"2024-09-17T09:26:46.476496+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:56.753897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757220+0000","last_change":"2024-09-17T09:26:20.778571+0000","last_active":"2024-09-17T09:26:46.757220+0000","last_peered":"2024-09-17T09:26:46.757220+0000","last_clean":"2024-09-17T09:26:46.757220+0000","last_became_active":"2024-09-17T09:26:20.778254+0000","last_became_peered":"2024-09-17T09:26:20.778254+0000","last_unstale":"2024-09-17T09:26:46.757220+0000","last_undegraded":"2024-09-17T09:26:46.757220+0000","last_fullsized":"2024-09-17T09:26:46.757220+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:46:05.622258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474264+0000","last_change":"2024-09-17T09:26:16.742822+0000","last_active":"2024-09-17T09:26:46.474264+0000","last_peered":"2024-09-17T09:26:46.474264+0000","last_clean":"2024-09-17T09:26:46.474264+0000","last_became_active":"2024-09-17T09:26:16.742546+0000","last_became_peered":"2024-09-17T09:26:16.742546+0000","last_unstale":"2024-09-17T09:26:46.474264+0000","last_undegraded":"2024-09-17T09:26:46.474264+0000","last_fullsized":"2024-09-17T09:26:46.474264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:08:07.193982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.029055+0000","last_change":"2024-09-17T09:26:47.028633+0000","last_active":"2024-09-17T09:26:47.029055+0000","last_peered":"2024-09-17T09:26:47.029055+0000","last_clean":"2024-09-17T09:26:47.029055+0000","last_became_active":"2024-09-17T09:26:47.028221+0000","last_became_peered":"2024-09-17T09:26:47.028221+0000","last_unstale":"2024-09-17T09:26:47.029055+0000","last_undegraded":"2024-09-17T09:26:47.029055+0000","last_fullsized":"2024-09-17T09:26:47.029055+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:00.419613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'188","reported_seq":336,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:00.590292+0000","last_change":"2024-09-17T09:26:05.194885+0000","last_active":"2024-09-17T09:27:00.590292+0000","last_peered":"2024-09-17T09:27:00.590292+0000","last_clean":"2024-09-17T09:27:00.590292+0000","last_became_active":"2024-09-17T09:26:05.194420+0000","last_became_peered":"2024-09-17T09:26:05.194420+0000","last_unstale":"2024-09-17T09:27:00.590292+0000","last_undegraded":"2024-09-17T09:27:00.590292+0000","last_fullsized":"2024-09-17T09:27:00.590292+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:23:58.802554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:23:58.802554+0000","last_clean_scrub_stamp":"2024-09-17T09:23:58.802554+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:08.801262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473926+0000","last_change":"2024-09-17T09:26:18.767051+0000","last_active":"2024-09-17T09:26:46.473926+0000","last_peered":"2024-09-17T09:26:46.473926+0000","last_clean":"2024-09-17T09:26:46.473926+0000","last_became_active":"2024-09-17T09:26:18.766746+0000","last_became_peered":"2024-09-17T09:26:18.766746+0000","last_unstale":"2024-09-17T09:26:46.473926+0000","last_undegraded":"2024-09-17T09:26:46.473926+0000","last_fullsized":"2024-09-17T09:26:46.473926+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:23.507097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476388+0000","last_change":"2024-09-17T09:26:20.777416+0000","last_active":"2024-09-17T09:26:46.476388+0000","last_peered":"2024-09-17T09:26:46.476388+0000","last_clean":"2024-09-17T09:26:46.476388+0000","last_became_active":"2024-09-17T09:26:20.777096+0000","last_became_peered":"2024-09-17T09:26:20.777096+0000","last_unstale":"2024-09-17T09:26:46.476388+0000","last_undegraded":"2024-09-17T09:26:46.476388+0000","last_fullsized":"2024-09-17T09:26:46.476388+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:09.613262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"67'17","reported_seq":47,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474347+0000","last_change":"2024-09-17T09:26:16.741985+0000","last_active":"2024-09-17T09:26:46.474347+0000","last_peered":"2024-09-17T09:26:46.474347+0000","last_clean":"2024-09-17T09:26:46.474347+0000","last_became_active":"2024-09-17T09:26:16.741689+0000","last_became_peered":"2024-09-17T09:26:16.741689+0000","last_unstale":"2024-09-17T09:26:46.474347+0000","last_undegraded":"2024-09-17T09:26:46.474347+0000","last_fullsized":"2024-09-17T09:26:46.474347+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:55.034586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027126+0000","last_change":"2024-09-17T09:26:14.729373+0000","last_active":"2024-09-17T09:26:47.027126+0000","last_peered":"2024-09-17T09:26:47.027126+0000","last_clean":"2024-09-17T09:26:47.027126+0000","last_became_active":"2024-09-17T09:26:14.729081+0000","last_became_peered":"2024-09-17T09:26:14.729081+0000","last_unstale":"2024-09-17T09:26:47.027126+0000","last_undegraded":"2024-09-17T09:26:47.027126+0000","last_fullsized":"2024-09-17T09:26:47.027126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:46.245436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027583+0000","last_change":"2024-09-17T09:26:18.760963+0000","last_active":"2024-09-17T09:26:47.027583+0000","last_peered":"2024-09-17T09:26:47.027583+0000","last_clean":"2024-09-17T09:26:47.027583+0000","last_became_active":"2024-09-17T09:26:18.760655+0000","last_became_peered":"2024-09-17T09:26:18.760655+0000","last_unstale":"2024-09-17T09:26:47.027583+0000","last_undegraded":"2024-09-17T09:26:47.027583+0000","last_fullsized":"2024-09-17T09:26:47.027583+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:11.885213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473655+0000","last_change":"2024-09-17T09:26:20.780780+0000","last_active":"2024-09-17T09:26:46.473655+0000","last_peered":"2024-09-17T09:26:46.473655+0000","last_clean":"2024-09-17T09:26:46.473655+0000","last_became_active":"2024-09-17T09:26:20.780525+0000","last_became_peered":"2024-09-17T09:26:20.780525+0000","last_unstale":"2024-09-17T09:26:46.473655+0000","last_undegraded":"2024-09-17T09:26:46.473655+0000","last_fullsized":"2024-09-17T09:26:46.473655+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:30.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476322+0000","last_change":"2024-09-17T09:26:16.743425+0000","last_active":"2024-09-17T09:26:46.476322+0000","last_peered":"2024-09-17T09:26:46.476322+0000","last_clean":"2024-09-17T09:26:46.476322+0000","last_became_active":"2024-09-17T09:26:16.743182+0000","last_became_peered":"2024-09-17T09:26:16.743182+0000","last_unstale":"2024-09-17T09:26:46.476322+0000","last_undegraded":"2024-09-17T09:26:46.476322+0000","last_fullsized":"2024-09-17T09:26:46.476322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:56.606826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756662+0000","last_change":"2024-09-17T09:26:14.724679+0000","last_active":"2024-09-17T09:26:46.756662+0000","last_peered":"2024-09-17T09:26:46.756662+0000","last_clean":"2024-09-17T09:26:46.756662+0000","last_became_active":"2024-09-17T09:26:14.724432+0000","last_became_peered":"2024-09-17T09:26:14.724432+0000","last_unstale":"2024-09-17T09:26:46.756662+0000","last_undegraded":"2024-09-17T09:26:46.756662+0000","last_fullsized":"2024-09-17T09:26:46.756662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:36:47.957658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027485+0000","last_change":"2024-09-17T09:26:18.760179+0000","last_active":"2024-09-17T09:26:47.027485+0000","last_peered":"2024-09-17T09:26:47.027485+0000","last_clean":"2024-09-17T09:26:47.027485+0000","last_became_active":"2024-09-17T09:26:18.759842+0000","last_became_peered":"2024-09-17T09:26:18.759842+0000","last_unstale":"2024-09-17T09:26:47.027485+0000","last_undegraded":"2024-09-17T09:26:47.027485+0000","last_fullsized":"2024-09-17T09:26:47.027485+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:24:29.915445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273395+0000","last_change":"2024-09-17T09:26:20.779805+0000","last_active":"2024-09-17T09:26:47.273395+0000","last_peered":"2024-09-17T09:26:47.273395+0000","last_clean":"2024-09-17T09:26:47.273395+0000","last_became_active":"2024-09-17T09:26:20.779489+0000","last_became_peered":"2024-09-17T09:26:20.779489+0000","last_unstale":"2024-09-17T09:26:47.273395+0000","last_undegraded":"2024-09-17T09:26:47.273395+0000","last_fullsized":"2024-09-17T09:26:47.273395+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:33.662456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474114+0000","last_change":"2024-09-17T09:26:16.744297+0000","last_active":"2024-09-17T09:26:46.474114+0000","last_peered":"2024-09-17T09:26:46.474114+0000","last_clean":"2024-09-17T09:26:46.474114+0000","last_became_active":"2024-09-17T09:26:16.744030+0000","last_became_peered":"2024-09-17T09:26:16.744030+0000","last_unstale":"2024-09-17T09:26:46.474114+0000","last_undegraded":"2024-09-17T09:26:46.474114+0000","last_fullsized":"2024-09-17T09:26:46.474114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:41.597190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027125+0000","last_change":"2024-09-17T09:26:14.723780+0000","last_active":"2024-09-17T09:26:47.027125+0000","last_peered":"2024-09-17T09:26:47.027125+0000","last_clean":"2024-09-17T09:26:47.027125+0000","last_became_active":"2024-09-17T09:26:14.723501+0000","last_became_peered":"2024-09-17T09:26:14.723501+0000","last_unstale":"2024-09-17T09:26:47.027125+0000","last_undegraded":"2024-09-17T09:26:47.027125+0000","last_fullsized":"2024-09-17T09:26:47.027125+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:34.372643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"67'11","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:47.947470+0000","last_change":"2024-09-17T09:26:18.764303+0000","last_active":"2024-09-17T09:27:47.947470+0000","last_peered":"2024-09-17T09:27:47.947470+0000","last_clean":"2024-09-17T09:27:47.947470+0000","last_became_active":"2024-09-17T09:26:18.764024+0000","last_became_peered":"2024-09-17T09:26:18.764024+0000","last_unstale":"2024-09-17T09:27:47.947470+0000","last_undegraded":"2024-09-17T09:27:47.947470+0000","last_fullsized":"2024-09-17T09:27:47.947470+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:40:59.047020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.475898+0000","last_change":"2024-09-17T09:26:21.247482+0000","last_active":"2024-09-17T09:26:46.475898+0000","last_peered":"2024-09-17T09:26:46.475898+0000","last_clean":"2024-09-17T09:26:46.475898+0000","last_became_active":"2024-09-17T09:26:21.247125+0000","last_became_peered":"2024-09-17T09:26:21.247125+0000","last_unstale":"2024-09-17T09:26:46.475898+0000","last_undegraded":"2024-09-17T09:26:46.475898+0000","last_fullsized":"2024-09-17T09:26:46.475898+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:58.759643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474391+0000","last_change":"2024-09-17T09:26:16.742841+0000","last_active":"2024-09-17T09:26:46.474391+0000","last_peered":"2024-09-17T09:26:46.474391+0000","last_clean":"2024-09-17T09:26:46.474391+0000","last_became_active":"2024-09-17T09:26:16.742582+0000","last_became_peered":"2024-09-17T09:26:16.742582+0000","last_unstale":"2024-09-17T09:26:46.474391+0000","last_undegraded":"2024-09-17T09:26:46.474391+0000","last_fullsized":"2024-09-17T09:26:46.474391+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:38:28.233133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.870322+0000","last_change":"2024-09-17T09:26:14.724221+0000","last_active":"2024-09-17T09:26:46.870322+0000","last_peered":"2024-09-17T09:26:46.870322+0000","last_clean":"2024-09-17T09:26:46.870322+0000","last_became_active":"2024-09-17T09:26:14.723921+0000","last_became_peered":"2024-09-17T09:26:14.723921+0000","last_unstale":"2024-09-17T09:26:46.870322+0000","last_undegraded":"2024-09-17T09:26:46.870322+0000","last_fullsized":"2024-09-17T09:26:46.870322+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:27.820473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027330+0000","last_change":"2024-09-17T09:26:18.762212+0000","last_active":"2024-09-17T09:26:47.027330+0000","last_peered":"2024-09-17T09:26:47.027330+0000","last_clean":"2024-09-17T09:26:47.027330+0000","last_became_active":"2024-09-17T09:26:18.761953+0000","last_became_peered":"2024-09-17T09:26:18.761953+0000","last_unstale":"2024-09-17T09:26:47.027330+0000","last_undegraded":"2024-09-17T09:26:47.027330+0000","last_fullsized":"2024-09-17T09:26:47.027330+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:51:46.020357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474327+0000","last_change":"2024-09-17T09:26:20.781461+0000","last_active":"2024-09-17T09:26:46.474327+0000","last_peered":"2024-09-17T09:26:46.474327+0000","last_clean":"2024-09-17T09:26:46.474327+0000","last_became_active":"2024-09-17T09:26:20.781087+0000","last_became_peered":"2024-09-17T09:26:20.781087+0000","last_unstale":"2024-09-17T09:26:46.474327+0000","last_undegraded":"2024-09-17T09:26:46.474327+0000","last_fullsized":"2024-09-17T09:26:46.474327+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:11.321299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473635+0000","last_change":"2024-09-17T09:26:16.746278+0000","last_active":"2024-09-17T09:26:46.473635+0000","last_peered":"2024-09-17T09:26:46.473635+0000","last_clean":"2024-09-17T09:26:46.473635+0000","last_became_active":"2024-09-17T09:26:16.744810+0000","last_became_peered":"2024-09-17T09:26:16.744810+0000","last_unstale":"2024-09-17T09:26:46.473635+0000","last_undegraded":"2024-09-17T09:26:46.473635+0000","last_fullsized":"2024-09-17T09:26:46.473635+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:20:35.063630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027450+0000","last_change":"2024-09-17T09:26:14.729692+0000","last_active":"2024-09-17T09:26:47.027450+0000","last_peered":"2024-09-17T09:26:47.027450+0000","last_clean":"2024-09-17T09:26:47.027450+0000","last_became_active":"2024-09-17T09:26:14.728873+0000","last_became_peered":"2024-09-17T09:26:14.728873+0000","last_unstale":"2024-09-17T09:26:47.027450+0000","last_undegraded":"2024-09-17T09:26:47.027450+0000","last_fullsized":"2024-09-17T09:26:47.027450+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:27:58.498470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474992+0000","last_change":"2024-09-17T09:26:18.764593+0000","last_active":"2024-09-17T09:26:46.474992+0000","last_peered":"2024-09-17T09:26:46.474992+0000","last_clean":"2024-09-17T09:26:46.474992+0000","last_became_active":"2024-09-17T09:26:18.764115+0000","last_became_peered":"2024-09-17T09:26:18.764115+0000","last_unstale":"2024-09-17T09:26:46.474992+0000","last_undegraded":"2024-09-17T09:26:46.474992+0000","last_fullsized":"2024-09-17T09:26:46.474992+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:56:07.366252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"67'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027574+0000","last_change":"2024-09-17T09:26:20.773479+0000","last_active":"2024-09-17T09:26:47.027574+0000","last_peered":"2024-09-17T09:26:47.027574+0000","last_clean":"2024-09-17T09:26:47.027574+0000","last_became_active":"2024-09-17T09:26:20.772814+0000","last_became_peered":"2024-09-17T09:26:20.772814+0000","last_unstale":"2024-09-17T09:26:47.027574+0000","last_undegraded":"2024-09-17T09:26:47.027574+0000","last_fullsized":"2024-09-17T09:26:47.027574+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:34:32.784044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"67'4","reported_seq":25,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223569+0000","last_change":"2024-09-17T09:26:16.747139+0000","last_active":"2024-09-17T09:26:47.223569+0000","last_peered":"2024-09-17T09:26:47.223569+0000","last_clean":"2024-09-17T09:26:47.223569+0000","last_became_active":"2024-09-17T09:26:16.746854+0000","last_became_peered":"2024-09-17T09:26:16.746854+0000","last_unstale":"2024-09-17T09:26:47.223569+0000","last_undegraded":"2024-09-17T09:26:47.223569+0000","last_fullsized":"2024-09-17T09:26:47.223569+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:37:45.212652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.224025+0000","last_change":"2024-09-17T09:26:14.720744+0000","last_active":"2024-09-17T09:26:47.224025+0000","last_peered":"2024-09-17T09:26:47.224025+0000","last_clean":"2024-09-17T09:26:47.224025+0000","last_became_active":"2024-09-17T09:26:14.720014+0000","last_became_peered":"2024-09-17T09:26:14.720014+0000","last_unstale":"2024-09-17T09:26:47.224025+0000","last_undegraded":"2024-09-17T09:26:47.224025+0000","last_fullsized":"2024-09-17T09:26:47.224025+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:45:19.672879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476333+0000","last_change":"2024-09-17T09:26:18.762346+0000","last_active":"2024-09-17T09:26:46.476333+0000","last_peered":"2024-09-17T09:26:46.476333+0000","last_clean":"2024-09-17T09:26:46.476333+0000","last_became_active":"2024-09-17T09:26:18.762019+0000","last_became_peered":"2024-09-17T09:26:18.762019+0000","last_unstale":"2024-09-17T09:26:46.476333+0000","last_undegraded":"2024-09-17T09:26:46.476333+0000","last_fullsized":"2024-09-17T09:26:46.476333+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:47.565382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473968+0000","last_change":"2024-09-17T09:26:21.250107+0000","last_active":"2024-09-17T09:26:46.473968+0000","last_peered":"2024-09-17T09:26:46.473968+0000","last_clean":"2024-09-17T09:26:46.473968+0000","last_became_active":"2024-09-17T09:26:21.249800+0000","last_became_peered":"2024-09-17T09:26:21.249800+0000","last_unstale":"2024-09-17T09:26:46.473968+0000","last_undegraded":"2024-09-17T09:26:46.473968+0000","last_fullsized":"2024-09-17T09:26:46.473968+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:58:08.827879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473881+0000","last_change":"2024-09-17T09:26:16.812511+0000","last_active":"2024-09-17T09:26:46.473881+0000","last_peered":"2024-09-17T09:26:46.473881+0000","last_clean":"2024-09-17T09:26:46.473881+0000","last_became_active":"2024-09-17T09:26:16.812239+0000","last_became_peered":"2024-09-17T09:26:16.812239+0000","last_unstale":"2024-09-17T09:26:46.473881+0000","last_undegraded":"2024-09-17T09:26:46.473881+0000","last_fullsized":"2024-09-17T09:26:46.473881+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:12.466728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476013+0000","last_change":"2024-09-17T09:26:14.723105+0000","last_active":"2024-09-17T09:26:46.476013+0000","last_peered":"2024-09-17T09:26:46.476013+0000","last_clean":"2024-09-17T09:26:46.476013+0000","last_became_active":"2024-09-17T09:26:14.722780+0000","last_became_peered":"2024-09-17T09:26:14.722780+0000","last_unstale":"2024-09-17T09:26:46.476013+0000","last_undegraded":"2024-09-17T09:26:46.476013+0000","last_fullsized":"2024-09-17T09:26:46.476013+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.414459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:47.947474+0000","last_change":"2024-09-17T09:26:18.765530+0000","last_active":"2024-09-17T09:27:47.947474+0000","last_peered":"2024-09-17T09:27:47.947474+0000","last_clean":"2024-09-17T09:27:47.947474+0000","last_became_active":"2024-09-17T09:26:18.765271+0000","last_became_peered":"2024-09-17T09:26:18.765271+0000","last_unstale":"2024-09-17T09:27:47.947474+0000","last_undegraded":"2024-09-17T09:27:47.947474+0000","last_fullsized":"2024-09-17T09:27:47.947474+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:21.154061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273795+0000","last_change":"2024-09-17T09:26:20.780262+0000","last_active":"2024-09-17T09:26:47.273795+0000","last_peered":"2024-09-17T09:26:47.273795+0000","last_clean":"2024-09-17T09:26:47.273795+0000","last_became_active":"2024-09-17T09:26:20.779910+0000","last_became_peered":"2024-09-17T09:26:20.779910+0000","last_unstale":"2024-09-17T09:26:47.273795+0000","last_undegraded":"2024-09-17T09:26:47.273795+0000","last_fullsized":"2024-09-17T09:26:47.273795+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:27.752379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273397+0000","last_change":"2024-09-17T09:26:16.744738+0000","last_active":"2024-09-17T09:26:47.273397+0000","last_peered":"2024-09-17T09:26:47.273397+0000","last_clean":"2024-09-17T09:26:47.273397+0000","last_became_active":"2024-09-17T09:26:16.744456+0000","last_became_peered":"2024-09-17T09:26:16.744456+0000","last_unstale":"2024-09-17T09:26:47.273397+0000","last_undegraded":"2024-09-17T09:26:47.273397+0000","last_fullsized":"2024-09-17T09:26:47.273397+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:03.571673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.272774+0000","last_change":"2024-09-17T09:26:14.720843+0000","last_active":"2024-09-17T09:26:47.272774+0000","last_peered":"2024-09-17T09:26:47.272774+0000","last_clean":"2024-09-17T09:26:47.272774+0000","last_became_active":"2024-09-17T09:26:14.720564+0000","last_became_peered":"2024-09-17T09:26:14.720564+0000","last_unstale":"2024-09-17T09:26:47.272774+0000","last_undegraded":"2024-09-17T09:26:47.272774+0000","last_fullsized":"2024-09-17T09:26:47.272774+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:01.314290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:47.947614+0000","last_change":"2024-09-17T09:26:18.760606+0000","last_active":"2024-09-17T09:27:47.947614+0000","last_peered":"2024-09-17T09:27:47.947614+0000","last_clean":"2024-09-17T09:27:47.947614+0000","last_became_active":"2024-09-17T09:26:18.760123+0000","last_became_peered":"2024-09-17T09:26:18.760123+0000","last_unstale":"2024-09-17T09:27:47.947614+0000","last_undegraded":"2024-09-17T09:27:47.947614+0000","last_fullsized":"2024-09-17T09:27:47.947614+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:42.478141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871716+0000","last_change":"2024-09-17T09:26:20.780844+0000","last_active":"2024-09-17T09:26:46.871716+0000","last_peered":"2024-09-17T09:26:46.871716+0000","last_clean":"2024-09-17T09:26:46.871716+0000","last_became_active":"2024-09-17T09:26:20.780542+0000","last_became_peered":"2024-09-17T09:26:20.780542+0000","last_unstale":"2024-09-17T09:26:46.871716+0000","last_undegraded":"2024-09-17T09:26:46.871716+0000","last_fullsized":"2024-09-17T09:26:46.871716+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:29.566402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474301+0000","last_change":"2024-09-17T09:26:16.743577+0000","last_active":"2024-09-17T09:26:46.474301+0000","last_peered":"2024-09-17T09:26:46.474301+0000","last_clean":"2024-09-17T09:26:46.474301+0000","last_became_active":"2024-09-17T09:26:16.743284+0000","last_became_peered":"2024-09-17T09:26:16.743284+0000","last_unstale":"2024-09-17T09:26:46.474301+0000","last_undegraded":"2024-09-17T09:26:46.474301+0000","last_fullsized":"2024-09-17T09:26:46.474301+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:25.350577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223270+0000","last_change":"2024-09-17T09:26:14.726094+0000","last_active":"2024-09-17T09:26:47.223270+0000","last_peered":"2024-09-17T09:26:47.223270+0000","last_clean":"2024-09-17T09:26:47.223270+0000","last_became_active":"2024-09-17T09:26:14.725879+0000","last_became_peered":"2024-09-17T09:26:14.725879+0000","last_unstale":"2024-09-17T09:26:47.223270+0000","last_undegraded":"2024-09-17T09:26:47.223270+0000","last_fullsized":"2024-09-17T09:26:47.223270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:52.795278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474424+0000","last_change":"2024-09-17T09:26:18.760740+0000","last_active":"2024-09-17T09:26:46.474424+0000","last_peered":"2024-09-17T09:26:46.474424+0000","last_clean":"2024-09-17T09:26:46.474424+0000","last_became_active":"2024-09-17T09:26:18.760436+0000","last_became_peered":"2024-09-17T09:26:18.760436+0000","last_unstale":"2024-09-17T09:26:46.474424+0000","last_undegraded":"2024-09-17T09:26:46.474424+0000","last_fullsized":"2024-09-17T09:26:46.474424+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:17:18.481597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.272994+0000","last_change":"2024-09-17T09:26:20.775652+0000","last_active":"2024-09-17T09:26:47.272994+0000","last_peered":"2024-09-17T09:26:47.272994+0000","last_clean":"2024-09-17T09:26:47.272994+0000","last_became_active":"2024-09-17T09:26:20.775088+0000","last_became_peered":"2024-09-17T09:26:20.775088+0000","last_unstale":"2024-09-17T09:26:47.272994+0000","last_undegraded":"2024-09-17T09:26:47.272994+0000","last_fullsized":"2024-09-17T09:26:47.272994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:03.697272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871376+0000","last_change":"2024-09-17T09:26:16.748842+0000","last_active":"2024-09-17T09:26:46.871376+0000","last_peered":"2024-09-17T09:26:46.871376+0000","last_clean":"2024-09-17T09:26:46.871376+0000","last_became_active":"2024-09-17T09:26:16.748458+0000","last_became_peered":"2024-09-17T09:26:16.748458+0000","last_unstale":"2024-09-17T09:26:46.871376+0000","last_undegraded":"2024-09-17T09:26:46.871376+0000","last_fullsized":"2024-09-17T09:26:46.871376+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:21.515972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756731+0000","last_change":"2024-09-17T09:26:14.728073+0000","last_active":"2024-09-17T09:26:46.756731+0000","last_peered":"2024-09-17T09:26:46.756731+0000","last_clean":"2024-09-17T09:26:46.756731+0000","last_became_active":"2024-09-17T09:26:14.727752+0000","last_became_peered":"2024-09-17T09:26:14.727752+0000","last_unstale":"2024-09-17T09:26:46.756731+0000","last_undegraded":"2024-09-17T09:26:46.756731+0000","last_fullsized":"2024-09-17T09:26:46.756731+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:56.351066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757103+0000","last_change":"2024-09-17T09:26:18.766856+0000","last_active":"2024-09-17T09:26:46.757103+0000","last_peered":"2024-09-17T09:26:46.757103+0000","last_clean":"2024-09-17T09:26:46.757103+0000","last_became_active":"2024-09-17T09:26:18.766558+0000","last_became_peered":"2024-09-17T09:26:18.766558+0000","last_unstale":"2024-09-17T09:26:46.757103+0000","last_undegraded":"2024-09-17T09:26:46.757103+0000","last_fullsized":"2024-09-17T09:26:46.757103+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:24.514423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473939+0000","last_change":"2024-09-17T09:26:20.783056+0000","last_active":"2024-09-17T09:26:46.473939+0000","last_peered":"2024-09-17T09:26:46.473939+0000","last_clean":"2024-09-17T09:26:46.473939+0000","last_became_active":"2024-09-17T09:26:20.782769+0000","last_became_peered":"2024-09-17T09:26:20.782769+0000","last_unstale":"2024-09-17T09:26:46.473939+0000","last_undegraded":"2024-09-17T09:26:46.473939+0000","last_fullsized":"2024-09-17T09:26:46.473939+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:48:26.748095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"67'6","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273979+0000","last_change":"2024-09-17T09:26:16.742808+0000","last_active":"2024-09-17T09:26:47.273979+0000","last_peered":"2024-09-17T09:26:47.273979+0000","last_clean":"2024-09-17T09:26:47.273979+0000","last_became_active":"2024-09-17T09:26:16.742540+0000","last_became_peered":"2024-09-17T09:26:16.742540+0000","last_unstale":"2024-09-17T09:26:47.273979+0000","last_undegraded":"2024-09-17T09:26:47.273979+0000","last_fullsized":"2024-09-17T09:26:47.273979+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:06:21.273210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"60'1","reported_seq":26,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476442+0000","last_change":"2024-09-17T09:26:14.720259+0000","last_active":"2024-09-17T09:26:46.476442+0000","last_peered":"2024-09-17T09:26:46.476442+0000","last_clean":"2024-09-17T09:26:46.476442+0000","last_became_active":"2024-09-17T09:26:14.719991+0000","last_became_peered":"2024-09-17T09:26:14.719991+0000","last_unstale":"2024-09-17T09:26:46.476442+0000","last_undegraded":"2024-09-17T09:26:46.476442+0000","last_fullsized":"2024-09-17T09:26:46.476442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:21.002411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223891+0000","last_change":"2024-09-17T09:26:18.767543+0000","last_active":"2024-09-17T09:26:47.223891+0000","last_peered":"2024-09-17T09:26:47.223891+0000","last_clean":"2024-09-17T09:26:47.223891+0000","last_became_active":"2024-09-17T09:26:18.767244+0000","last_became_peered":"2024-09-17T09:26:18.767244+0000","last_unstale":"2024-09-17T09:26:47.223891+0000","last_undegraded":"2024-09-17T09:26:47.223891+0000","last_fullsized":"2024-09-17T09:26:47.223891+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:42:38.686983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473470+0000","last_change":"2024-09-17T09:26:20.774288+0000","last_active":"2024-09-17T09:26:46.473470+0000","last_peered":"2024-09-17T09:26:46.473470+0000","last_clean":"2024-09-17T09:26:46.473470+0000","last_became_active":"2024-09-17T09:26:20.773915+0000","last_became_peered":"2024-09-17T09:26:20.773915+0000","last_unstale":"2024-09-17T09:26:46.473470+0000","last_undegraded":"2024-09-17T09:26:46.473470+0000","last_fullsized":"2024-09-17T09:26:46.473470+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:45.049464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476358+0000","last_change":"2024-09-17T09:26:16.743528+0000","last_active":"2024-09-17T09:26:46.476358+0000","last_peered":"2024-09-17T09:26:46.476358+0000","last_clean":"2024-09-17T09:26:46.476358+0000","last_became_active":"2024-09-17T09:26:16.743065+0000","last_became_peered":"2024-09-17T09:26:16.743065+0000","last_unstale":"2024-09-17T09:26:46.476358+0000","last_undegraded":"2024-09-17T09:26:46.476358+0000","last_fullsized":"2024-09-17T09:26:46.476358+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:02:33.861468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223520+0000","last_change":"2024-09-17T09:26:14.725546+0000","last_active":"2024-09-17T09:26:47.223520+0000","last_peered":"2024-09-17T09:26:47.223520+0000","last_clean":"2024-09-17T09:26:47.223520+0000","last_became_active":"2024-09-17T09:26:14.725285+0000","last_became_peered":"2024-09-17T09:26:14.725285+0000","last_unstale":"2024-09-17T09:26:47.223520+0000","last_undegraded":"2024-09-17T09:26:47.223520+0000","last_fullsized":"2024-09-17T09:26:47.223520+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:28.057127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473325+0000","last_change":"2024-09-17T09:26:18.765072+0000","last_active":"2024-09-17T09:26:46.473325+0000","last_peered":"2024-09-17T09:26:46.473325+0000","last_clean":"2024-09-17T09:26:46.473325+0000","last_became_active":"2024-09-17T09:26:18.764824+0000","last_became_peered":"2024-09-17T09:26:18.764824+0000","last_unstale":"2024-09-17T09:26:46.473325+0000","last_undegraded":"2024-09-17T09:26:46.473325+0000","last_fullsized":"2024-09-17T09:26:46.473325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:17:17.825245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474516+0000","last_change":"2024-09-17T09:26:21.248366+0000","last_active":"2024-09-17T09:26:46.474516+0000","last_peered":"2024-09-17T09:26:46.474516+0000","last_clean":"2024-09-17T09:26:46.474516+0000","last_became_active":"2024-09-17T09:26:21.247942+0000","last_became_peered":"2024-09-17T09:26:21.247942+0000","last_unstale":"2024-09-17T09:26:46.474516+0000","last_undegraded":"2024-09-17T09:26:46.474516+0000","last_fullsized":"2024-09-17T09:26:46.474516+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:52.064235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473876+0000","last_change":"2024-09-17T09:26:20.781221+0000","last_active":"2024-09-17T09:26:46.473876+0000","last_peered":"2024-09-17T09:26:46.473876+0000","last_clean":"2024-09-17T09:26:46.473876+0000","last_became_active":"2024-09-17T09:26:20.780549+0000","last_became_peered":"2024-09-17T09:26:20.780549+0000","last_unstale":"2024-09-17T09:26:46.473876+0000","last_undegraded":"2024-09-17T09:26:46.473876+0000","last_fullsized":"2024-09-17T09:26:46.473876+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:11.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757016+0000","last_change":"2024-09-17T09:26:16.747672+0000","last_active":"2024-09-17T09:26:46.757016+0000","last_peered":"2024-09-17T09:26:46.757016+0000","last_clean":"2024-09-17T09:26:46.757016+0000","last_became_active":"2024-09-17T09:26:16.747379+0000","last_became_peered":"2024-09-17T09:26:16.747379+0000","last_unstale":"2024-09-17T09:26:46.757016+0000","last_undegraded":"2024-09-17T09:26:46.757016+0000","last_fullsized":"2024-09-17T09:26:46.757016+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:18.371366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.475608+0000","last_change":"2024-09-17T09:26:14.728190+0000","last_active":"2024-09-17T09:26:46.475608+0000","last_peered":"2024-09-17T09:26:46.475608+0000","last_clean":"2024-09-17T09:26:46.475608+0000","last_became_active":"2024-09-17T09:26:14.727874+0000","last_became_peered":"2024-09-17T09:26:14.727874+0000","last_unstale":"2024-09-17T09:26:46.475608+0000","last_undegraded":"2024-09-17T09:26:46.475608+0000","last_fullsized":"2024-09-17T09:26:46.475608+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:16:11.223541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"67'11","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:47.947471+0000","last_change":"2024-09-17T09:26:18.761105+0000","last_active":"2024-09-17T09:27:47.947471+0000","last_peered":"2024-09-17T09:27:47.947471+0000","last_clean":"2024-09-17T09:27:47.947471+0000","last_became_active":"2024-09-17T09:26:18.760694+0000","last_became_peered":"2024-09-17T09:26:18.760694+0000","last_unstale":"2024-09-17T09:27:47.947471+0000","last_undegraded":"2024-09-17T09:27:47.947471+0000","last_fullsized":"2024-09-17T09:27:47.947471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:30.174719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757378+0000","last_change":"2024-09-17T09:26:20.773420+0000","last_active":"2024-09-17T09:26:46.757378+0000","last_peered":"2024-09-17T09:26:46.757378+0000","last_clean":"2024-09-17T09:26:46.757378+0000","last_became_active":"2024-09-17T09:26:20.773133+0000","last_became_peered":"2024-09-17T09:26:20.773133+0000","last_unstale":"2024-09-17T09:26:46.757378+0000","last_undegraded":"2024-09-17T09:26:46.757378+0000","last_fullsized":"2024-09-17T09:26:46.757378+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:41.824017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473996+0000","last_change":"2024-09-17T09:26:16.742909+0000","last_active":"2024-09-17T09:26:46.473996+0000","last_peered":"2024-09-17T09:26:46.473996+0000","last_clean":"2024-09-17T09:26:46.473996+0000","last_became_active":"2024-09-17T09:26:16.742638+0000","last_became_peered":"2024-09-17T09:26:16.742638+0000","last_unstale":"2024-09-17T09:26:46.473996+0000","last_undegraded":"2024-09-17T09:26:46.473996+0000","last_fullsized":"2024-09-17T09:26:46.473996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:21:01.469453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474227+0000","last_change":"2024-09-17T09:26:14.728150+0000","last_active":"2024-09-17T09:26:46.474227+0000","last_peered":"2024-09-17T09:26:46.474227+0000","last_clean":"2024-09-17T09:26:46.474227+0000","last_became_active":"2024-09-17T09:26:14.727872+0000","last_became_peered":"2024-09-17T09:26:14.727872+0000","last_unstale":"2024-09-17T09:26:46.474227+0000","last_undegraded":"2024-09-17T09:26:46.474227+0000","last_fullsized":"2024-09-17T09:26:46.474227+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:05.726550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273795+0000","last_change":"2024-09-17T09:26:18.760419+0000","last_active":"2024-09-17T09:26:47.273795+0000","last_peered":"2024-09-17T09:26:47.273795+0000","last_clean":"2024-09-17T09:26:47.273795+0000","last_became_active":"2024-09-17T09:26:18.760107+0000","last_became_peered":"2024-09-17T09:26:18.760107+0000","last_unstale":"2024-09-17T09:26:47.273795+0000","last_undegraded":"2024-09-17T09:26:47.273795+0000","last_fullsized":"2024-09-17T09:26:47.273795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:46.209805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871489+0000","last_change":"2024-09-17T09:26:21.249932+0000","last_active":"2024-09-17T09:26:46.871489+0000","last_peered":"2024-09-17T09:26:46.871489+0000","last_clean":"2024-09-17T09:26:46.871489+0000","last_became_active":"2024-09-17T09:26:21.249590+0000","last_became_peered":"2024-09-17T09:26:21.249590+0000","last_unstale":"2024-09-17T09:26:46.871489+0000","last_undegraded":"2024-09-17T09:26:46.871489+0000","last_fullsized":"2024-09-17T09:26:46.871489+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:29:34.357306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223218+0000","last_change":"2024-09-17T09:26:14.722875+0000","last_active":"2024-09-17T09:26:47.223218+0000","last_peered":"2024-09-17T09:26:47.223218+0000","last_clean":"2024-09-17T09:26:47.223218+0000","last_became_active":"2024-09-17T09:26:14.722614+0000","last_became_peered":"2024-09-17T09:26:14.722614+0000","last_unstale":"2024-09-17T09:26:47.223218+0000","last_undegraded":"2024-09-17T09:26:47.223218+0000","last_fullsized":"2024-09-17T09:26:47.223218+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:32:36.214730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"67'5","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273675+0000","last_change":"2024-09-17T09:26:16.745725+0000","last_active":"2024-09-17T09:26:47.273675+0000","last_peered":"2024-09-17T09:26:47.273675+0000","last_clean":"2024-09-17T09:26:47.273675+0000","last_became_active":"2024-09-17T09:26:16.745523+0000","last_became_peered":"2024-09-17T09:26:16.745523+0000","last_unstale":"2024-09-17T09:26:47.273675+0000","last_undegraded":"2024-09-17T09:26:47.273675+0000","last_fullsized":"2024-09-17T09:26:47.273675+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:58.930892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757494+0000","last_change":"2024-09-17T09:26:18.764946+0000","last_active":"2024-09-17T09:26:46.757494+0000","last_peered":"2024-09-17T09:26:46.757494+0000","last_clean":"2024-09-17T09:26:46.757494+0000","last_became_active":"2024-09-17T09:26:18.764653+0000","last_became_peered":"2024-09-17T09:26:18.764653+0000","last_unstale":"2024-09-17T09:26:46.757494+0000","last_undegraded":"2024-09-17T09:26:46.757494+0000","last_fullsized":"2024-09-17T09:26:46.757494+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:04:59.060718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474914+0000","last_change":"2024-09-17T09:26:21.250320+0000","last_active":"2024-09-17T09:26:46.474914+0000","last_peered":"2024-09-17T09:26:46.474914+0000","last_clean":"2024-09-17T09:26:46.474914+0000","last_became_active":"2024-09-17T09:26:21.249995+0000","last_became_peered":"2024-09-17T09:26:21.249995+0000","last_unstale":"2024-09-17T09:26:46.474914+0000","last_undegraded":"2024-09-17T09:26:46.474914+0000","last_fullsized":"2024-09-17T09:26:46.474914+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:22:22.197956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474644+0000","last_change":"2024-09-17T09:26:14.728071+0000","last_active":"2024-09-17T09:26:46.474644+0000","last_peered":"2024-09-17T09:26:46.474644+0000","last_clean":"2024-09-17T09:26:46.474644+0000","last_became_active":"2024-09-17T09:26:14.727725+0000","last_became_peered":"2024-09-17T09:26:14.727725+0000","last_unstale":"2024-09-17T09:26:46.474644+0000","last_undegraded":"2024-09-17T09:26:46.474644+0000","last_fullsized":"2024-09-17T09:26:46.474644+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:31.258539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871459+0000","last_change":"2024-09-17T09:26:16.812384+0000","last_active":"2024-09-17T09:26:46.871459+0000","last_peered":"2024-09-17T09:26:46.871459+0000","last_clean":"2024-09-17T09:26:46.871459+0000","last_became_active":"2024-09-17T09:26:16.812072+0000","last_became_peered":"2024-09-17T09:26:16.812072+0000","last_unstale":"2024-09-17T09:26:46.871459+0000","last_undegraded":"2024-09-17T09:26:46.871459+0000","last_fullsized":"2024-09-17T09:26:46.871459+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:46:10.810106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871155+0000","last_change":"2024-09-17T09:26:18.759531+0000","last_active":"2024-09-17T09:26:46.871155+0000","last_peered":"2024-09-17T09:26:46.871155+0000","last_clean":"2024-09-17T09:26:46.871155+0000","last_became_active":"2024-09-17T09:26:18.758378+0000","last_became_peered":"2024-09-17T09:26:18.758378+0000","last_unstale":"2024-09-17T09:26:46.871155+0000","last_undegraded":"2024-09-17T09:26:46.871155+0000","last_fullsized":"2024-09-17T09:26:46.871155+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:24:26.845256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":701,"num_read_kb":458,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234009,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32488,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708568,"statfs":{"total":95990841344,"available":95957573632,"internally_reserved":0,"allocated":1409024,"data_stored":922276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.35799999999999998}]},{"osd":1,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.437}]},{"osd":2,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.52000000000000002}]},{"osd":6,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":4.2670000000000003,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.68000000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495646,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32452,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708604,"statfs":{"total":95990841344,"available":95957610496,"internally_reserved":0,"allocated":1368064,"data_stored":921553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":0.68700000000000006,"15min":0.57599999999999996},"min":{"1min":0.32800000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":16.181000000000001,"5min":16.181000000000001,"15min":16.181000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.502}]},{"osd":1,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":0.99399999999999999,"15min":0.93600000000000005},"min":{"1min":0.311,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":16.27,"5min":16.27,"15min":16.27},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.794,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":0.48999999999999999}]},{"osd":2,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51500000000000001,"15min":0.504},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.63400000000000001},"min":{"1min":0.39100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":0.46700000000000003}]},{"osd":3,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.59999999999999998,"15min":0.61899999999999999},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66000000000000003,"5min":3.3330000000000002,"15min":3.3330000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.52700000000000002,"15min":0.51500000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40000000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.496,"15min":0.51100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54800000000000004,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.43099999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.44400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.63400000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757284,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31888,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709168,"statfs":{"total":95990841344,"available":95958188032,"internally_reserved":0,"allocated":790528,"data_stored":331918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.65100000000000002,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.55800000000000005}]},{"osd":1,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.502,"15min":0.503},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.48299999999999998,"15min":0.47299999999999998},"min":{"1min":0.38400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.51400000000000001,"15min":0.496},"min":{"1min":0.40300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57099999999999995}]},{"osd":3,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52700000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.53700000000000003}]},{"osd":4,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.32000000000000001,"5min":0.251,"15min":0.251},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.502},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.40000000000000002,"15min":0.38800000000000001},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.39500000000000002}]},{"osd":6,"last update":"Tue Sep 17 09:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.55900000000000005,"15min":0.57699999999999996},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64300000000000002,"5min":2.496,"15min":2.496},"last":0.627},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45500000000000002,"15min":0.45300000000000001},"min":{"1min":0.35099999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66800000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":0.58299999999999996}]}]},{"osd":4,"up_from":30,"seq":128849018921,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31916,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709140,"statfs":{"total":95990841344,"available":95958159360,"internally_reserved":0,"allocated":819200,"data_stored":331226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.45200000000000001}]},{"osd":1,"last update":"Tue Sep 17 09:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.34999999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Sep 17 09:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.746,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.40100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47499999999999998}]},{"osd":3,"last update":"Tue Sep 17 09:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.496},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52100000000000002}]},{"osd":5,"last update":"Tue Sep 17 09:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.36899999999999999,"15min":0.35699999999999998},"min":{"1min":0.27700000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.42799999999999999,"15min":0.41099999999999998},"min":{"1min":0.316,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.41599999999999998}]},{"osd":6,"last update":"Tue Sep 17 09:27:39 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46100000000000002,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76900000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.46200000000000002}]},{"osd":7,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.48299999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713280,"statfs":{"total":95990841344,"available":95962398720,"internally_reserved":0,"allocated":774144,"data_stored":330655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.58899999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.39000000000000001}]},{"osd":1,"last update":"Tue Sep 17 09:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.41999999999999998,"15min":0.435},"min":{"1min":0.27000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.5,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.41199999999999998,"15min":0.42999999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.48099999999999998,"5min":0.625,"15min":0.625},"last":0.48999999999999999}]},{"osd":2,"last update":"Tue Sep 17 09:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.44800000000000001,"15min":0.46500000000000002},"min":{"1min":0.255,"5min":0.252,"15min":0.252},"max":{"1min":0.59199999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.28299999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.55600000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Tue Sep 17 09:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56100000000000005,"15min":0.57199999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.55000000000000004}]},{"osd":5,"last update":"Tue Sep 17 09:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.625}]},{"osd":6,"last update":"Tue Sep 17 09:27:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78100000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.67300000000000004}]},{"osd":7,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.59799999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713264,"statfs":{"total":95990841344,"available":95962382336,"internally_reserved":0,"allocated":790528,"data_stored":331206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34999999999999998,"15min":0.34200000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54400000000000004,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.375},{"interface":"front","average":{"1min":0.374,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.31900000000000001}]},{"osd":1,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.374,"15min":0.35399999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.54300000000000004,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.36799999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56599999999999995,"5min":0.63,"15min":0.63},"last":0.34200000000000003}]},{"osd":3,"last update":"Tue Sep 17 09:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.438,"15min":0.442},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55400000000000005,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.437,"15min":0.44500000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63400000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.59699999999999998}]},{"osd":4,"last update":"Tue Sep 17 09:27:51 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73699999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.56000000000000005,"15min":0.57899999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Sep 17 09:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.76000000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.499},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61499999999999999,"15min":0.626},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":1.012,"15min":1.012},"last":0.73699999999999999}]},{"osd":6,"last update":"Tue Sep 17 09:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.753,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.747,"5min":0.871,"15min":0.871},"last":0.53000000000000003}]},{"osd":7,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27796,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713260,"statfs":{"total":95990841344,"available":95962378240,"internally_reserved":0,"allocated":794624,"data_stored":331473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:42 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.32900000000000001,"15min":0.311},"min":{"1min":0.20599999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.45100000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"min":{"1min":0.223,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52000000000000002,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.442}]},{"osd":2,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.35399999999999998,"15min":0.34000000000000002},"min":{"1min":0.23100000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46999999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57299999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.36799999999999999}]},{"osd":3,"last update":"Tue Sep 17 09:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39600000000000002,"15min":0.40200000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41999999999999998,"15min":0.42199999999999999},"min":{"1min":0.29599999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.78700000000000003}]},{"osd":4,"last update":"Tue Sep 17 09:26:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.38100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.77400000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.80900000000000005}]},{"osd":5,"last update":"Tue Sep 17 09:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79600000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Sep 17 09:27:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.39500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.752}]},{"osd":7,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.76300000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28372,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712684,"statfs":{"total":95990841344,"available":95961788416,"internally_reserved":0,"allocated":1384448,"data_stored":921291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 09:27:36 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.33800000000000002,"15min":0.32300000000000001},"min":{"1min":0.23000000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.47699999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42999999999999999,"15min":0.39200000000000002},"min":{"1min":0.315,"5min":0.245,"15min":0.245},"max":{"1min":0.63900000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Sep 17 09:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.375,"15min":0.36499999999999999},"min":{"1min":0.28000000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.38},"min":{"1min":0.33600000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.505,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.44400000000000001}]},{"osd":3,"last update":"Tue Sep 17 09:27:28 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.441,"15min":0.44900000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.622,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.42099999999999999,"15min":0.42699999999999999},"min":{"1min":0.26200000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.309}]},{"osd":4,"last update":"Tue Sep 17 09:26:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53100000000000003,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.42399999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68200000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.622}]},{"osd":6,"last update":"Tue Sep 17 09:27:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55300000000000005,"15min":0.56000000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.629,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61299999999999999}]},{"osd":7,"last update":"Tue Sep 17 09:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.57199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T09:27:57.279 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-17T09:27:58.152 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:27:58.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:58 smithi148 ceph-mon[25650]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:58.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:58 smithi148 ceph-mon[32020]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:59.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:58 smithi157 ceph-mon[27402]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:27:59.731 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:27:59 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:27:59.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:27:59 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:27:59.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:27:59 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:27:59.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:27:59 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:28:00.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:00 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:00.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:00 smithi148 ceph-mon[32020]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:00.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:00 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:00.668 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:00 smithi148 ceph-mon[25650]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:00.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ::ffff:172.21.15.157 - - [17/Sep/2024:09:28:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:28:00.936 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:00 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:00.937 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:00 smithi157 ceph-mon[27402]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:01.807 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:28:01.809 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-09-17T09:28:02.246 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.246 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.247 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.247 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.247 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.247 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.247 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.247 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.247 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.247 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.247 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.247 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:02 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.366 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:02 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.366 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:02 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.366 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:02 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.366 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:02 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.367 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:02 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:02.367 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:02 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:03.079 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-09-17T09:28:00.539035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1059,"num_read_kb":902,"num_write":675,"num_write_kb":4025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":683,"ondisk_log_size":683,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":240480,"kb_used_data":7940,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":749687968,"statfs":{"total":767926730752,"available":767680479232,"internally_reserved":0,"allocated":8130560,"data_stored":4421598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002520"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474406+0000","last_change":"2024-09-17T09:26:21.248566+0000","last_active":"2024-09-17T09:26:46.474406+0000","last_peered":"2024-09-17T09:26:46.474406+0000","last_clean":"2024-09-17T09:26:46.474406+0000","last_became_active":"2024-09-17T09:26:21.248014+0000","last_became_peered":"2024-09-17T09:26:21.248014+0000","last_unstale":"2024-09-17T09:26:46.474406+0000","last_undegraded":"2024-09-17T09:26:46.474406+0000","last_fullsized":"2024-09-17T09:26:46.474406+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:20.315997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.272976+0000","last_change":"2024-09-17T09:26:14.723451+0000","last_active":"2024-09-17T09:26:47.272976+0000","last_peered":"2024-09-17T09:26:47.272976+0000","last_clean":"2024-09-17T09:26:47.272976+0000","last_became_active":"2024-09-17T09:26:14.723168+0000","last_became_peered":"2024-09-17T09:26:14.723168+0000","last_unstale":"2024-09-17T09:26:47.272976+0000","last_undegraded":"2024-09-17T09:26:47.272976+0000","last_fullsized":"2024-09-17T09:26:47.272976+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:09.679630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474927+0000","last_change":"2024-09-17T09:26:16.812395+0000","last_active":"2024-09-17T09:26:46.474927+0000","last_peered":"2024-09-17T09:26:46.474927+0000","last_clean":"2024-09-17T09:26:46.474927+0000","last_became_active":"2024-09-17T09:26:16.812016+0000","last_became_peered":"2024-09-17T09:26:16.812016+0000","last_unstale":"2024-09-17T09:26:46.474927+0000","last_undegraded":"2024-09-17T09:26:46.474927+0000","last_fullsized":"2024-09-17T09:26:46.474927+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:16.338194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.870767+0000","last_change":"2024-09-17T09:26:18.761963+0000","last_active":"2024-09-17T09:26:46.870767+0000","last_peered":"2024-09-17T09:26:46.870767+0000","last_clean":"2024-09-17T09:26:46.870767+0000","last_became_active":"2024-09-17T09:26:18.761670+0000","last_became_peered":"2024-09-17T09:26:18.761670+0000","last_unstale":"2024-09-17T09:26:46.870767+0000","last_undegraded":"2024-09-17T09:26:46.870767+0000","last_fullsized":"2024-09-17T09:26:46.870767+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:33.290068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474997+0000","last_change":"2024-09-17T09:26:14.726819+0000","last_active":"2024-09-17T09:26:46.474997+0000","last_peered":"2024-09-17T09:26:46.474997+0000","last_clean":"2024-09-17T09:26:46.474997+0000","last_became_active":"2024-09-17T09:26:14.726551+0000","last_became_peered":"2024-09-17T09:26:14.726551+0000","last_unstale":"2024-09-17T09:26:46.474997+0000","last_undegraded":"2024-09-17T09:26:46.474997+0000","last_fullsized":"2024-09-17T09:26:46.474997+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:36:14.494642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273979+0000","last_change":"2024-09-17T09:26:16.812463+0000","last_active":"2024-09-17T09:26:47.273979+0000","last_peered":"2024-09-17T09:26:47.273979+0000","last_clean":"2024-09-17T09:26:47.273979+0000","last_became_active":"2024-09-17T09:26:16.812187+0000","last_became_peered":"2024-09-17T09:26:16.812187+0000","last_unstale":"2024-09-17T09:26:47.273979+0000","last_undegraded":"2024-09-17T09:26:47.273979+0000","last_fullsized":"2024-09-17T09:26:47.273979+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:51.660034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756978+0000","last_change":"2024-09-17T09:26:18.761794+0000","last_active":"2024-09-17T09:26:46.756978+0000","last_peered":"2024-09-17T09:26:46.756978+0000","last_clean":"2024-09-17T09:26:46.756978+0000","last_became_active":"2024-09-17T09:26:18.761442+0000","last_became_peered":"2024-09-17T09:26:18.761442+0000","last_unstale":"2024-09-17T09:26:46.756978+0000","last_undegraded":"2024-09-17T09:26:46.756978+0000","last_fullsized":"2024-09-17T09:26:46.756978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:19.724993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871049+0000","last_change":"2024-09-17T09:26:20.769283+0000","last_active":"2024-09-17T09:26:46.871049+0000","last_peered":"2024-09-17T09:26:46.871049+0000","last_clean":"2024-09-17T09:26:46.871049+0000","last_became_active":"2024-09-17T09:26:20.768995+0000","last_became_peered":"2024-09-17T09:26:20.768995+0000","last_unstale":"2024-09-17T09:26:46.871049+0000","last_undegraded":"2024-09-17T09:26:46.871049+0000","last_fullsized":"2024-09-17T09:26:46.871049+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:10:33.969739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474394+0000","last_change":"2024-09-17T09:26:14.732633+0000","last_active":"2024-09-17T09:26:46.474394+0000","last_peered":"2024-09-17T09:26:46.474394+0000","last_clean":"2024-09-17T09:26:46.474394+0000","last_became_active":"2024-09-17T09:26:14.732292+0000","last_became_peered":"2024-09-17T09:26:14.732292+0000","last_unstale":"2024-09-17T09:26:46.474394+0000","last_undegraded":"2024-09-17T09:26:46.474394+0000","last_fullsized":"2024-09-17T09:26:46.474394+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:35:48.218891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476075+0000","last_change":"2024-09-17T09:26:16.745776+0000","last_active":"2024-09-17T09:26:46.476075+0000","last_peered":"2024-09-17T09:26:46.476075+0000","last_clean":"2024-09-17T09:26:46.476075+0000","last_became_active":"2024-09-17T09:26:16.745524+0000","last_became_peered":"2024-09-17T09:26:16.745524+0000","last_unstale":"2024-09-17T09:26:46.476075+0000","last_undegraded":"2024-09-17T09:26:46.476075+0000","last_fullsized":"2024-09-17T09:26:46.476075+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:54:01.177151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473646+0000","last_change":"2024-09-17T09:26:18.762720+0000","last_active":"2024-09-17T09:26:46.473646+0000","last_peered":"2024-09-17T09:26:46.473646+0000","last_clean":"2024-09-17T09:26:46.473646+0000","last_became_active":"2024-09-17T09:26:18.762400+0000","last_became_peered":"2024-09-17T09:26:18.762400+0000","last_unstale":"2024-09-17T09:26:46.473646+0000","last_undegraded":"2024-09-17T09:26:46.473646+0000","last_fullsized":"2024-09-17T09:26:46.473646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:39.153980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476154+0000","last_change":"2024-09-17T09:26:20.779004+0000","last_active":"2024-09-17T09:26:46.476154+0000","last_peered":"2024-09-17T09:26:46.476154+0000","last_clean":"2024-09-17T09:26:46.476154+0000","last_became_active":"2024-09-17T09:26:20.778421+0000","last_became_peered":"2024-09-17T09:26:20.778421+0000","last_unstale":"2024-09-17T09:26:46.476154+0000","last_undegraded":"2024-09-17T09:26:46.476154+0000","last_fullsized":"2024-09-17T09:26:46.476154+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:26.139869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474163+0000","last_change":"2024-09-17T09:26:14.728651+0000","last_active":"2024-09-17T09:26:46.474163+0000","last_peered":"2024-09-17T09:26:46.474163+0000","last_clean":"2024-09-17T09:26:46.474163+0000","last_became_active":"2024-09-17T09:26:14.728371+0000","last_became_peered":"2024-09-17T09:26:14.728371+0000","last_unstale":"2024-09-17T09:26:46.474163+0000","last_undegraded":"2024-09-17T09:26:46.474163+0000","last_fullsized":"2024-09-17T09:26:46.474163+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:40.009267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"67'12","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476313+0000","last_change":"2024-09-17T09:26:16.743382+0000","last_active":"2024-09-17T09:26:46.476313+0000","last_peered":"2024-09-17T09:26:46.476313+0000","last_clean":"2024-09-17T09:26:46.476313+0000","last_became_active":"2024-09-17T09:26:16.743123+0000","last_became_peered":"2024-09-17T09:26:16.743123+0000","last_unstale":"2024-09-17T09:26:46.476313+0000","last_undegraded":"2024-09-17T09:26:46.476313+0000","last_fullsized":"2024-09-17T09:26:46.476313+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:36:30.920679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.475789+0000","last_change":"2024-09-17T09:26:18.764655+0000","last_active":"2024-09-17T09:26:46.475789+0000","last_peered":"2024-09-17T09:26:46.475789+0000","last_clean":"2024-09-17T09:26:46.475789+0000","last_became_active":"2024-09-17T09:26:18.764365+0000","last_became_peered":"2024-09-17T09:26:18.764365+0000","last_unstale":"2024-09-17T09:26:46.475789+0000","last_undegraded":"2024-09-17T09:26:46.475789+0000","last_fullsized":"2024-09-17T09:26:46.475789+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:41:22.891498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273246+0000","last_change":"2024-09-17T09:26:20.774839+0000","last_active":"2024-09-17T09:26:47.273246+0000","last_peered":"2024-09-17T09:26:47.273246+0000","last_clean":"2024-09-17T09:26:47.273246+0000","last_became_active":"2024-09-17T09:26:20.774527+0000","last_became_peered":"2024-09-17T09:26:20.774527+0000","last_unstale":"2024-09-17T09:26:47.273246+0000","last_undegraded":"2024-09-17T09:26:47.273246+0000","last_fullsized":"2024-09-17T09:26:47.273246+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:27.642478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"67'19","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223854+0000","last_change":"2024-09-17T09:26:16.745913+0000","last_active":"2024-09-17T09:26:47.223854+0000","last_peered":"2024-09-17T09:26:47.223854+0000","last_clean":"2024-09-17T09:26:47.223854+0000","last_became_active":"2024-09-17T09:26:16.745655+0000","last_became_peered":"2024-09-17T09:26:16.745655+0000","last_unstale":"2024-09-17T09:26:47.223854+0000","last_undegraded":"2024-09-17T09:26:47.223854+0000","last_fullsized":"2024-09-17T09:26:47.223854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:33.404335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474155+0000","last_change":"2024-09-17T09:26:14.728051+0000","last_active":"2024-09-17T09:26:46.474155+0000","last_peered":"2024-09-17T09:26:46.474155+0000","last_clean":"2024-09-17T09:26:46.474155+0000","last_became_active":"2024-09-17T09:26:14.727749+0000","last_became_peered":"2024-09-17T09:26:14.727749+0000","last_unstale":"2024-09-17T09:26:46.474155+0000","last_undegraded":"2024-09-17T09:26:46.474155+0000","last_fullsized":"2024-09-17T09:26:46.474155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:52:08.725435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757283+0000","last_change":"2024-09-17T09:26:18.762298+0000","last_active":"2024-09-17T09:26:46.757283+0000","last_peered":"2024-09-17T09:26:46.757283+0000","last_clean":"2024-09-17T09:26:46.757283+0000","last_became_active":"2024-09-17T09:26:18.761985+0000","last_became_peered":"2024-09-17T09:26:18.761985+0000","last_unstale":"2024-09-17T09:26:46.757283+0000","last_undegraded":"2024-09-17T09:26:46.757283+0000","last_fullsized":"2024-09-17T09:26:46.757283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:45.470662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027869+0000","last_change":"2024-09-17T09:26:20.777657+0000","last_active":"2024-09-17T09:26:47.027869+0000","last_peered":"2024-09-17T09:26:47.027869+0000","last_clean":"2024-09-17T09:26:47.027869+0000","last_became_active":"2024-09-17T09:26:20.777315+0000","last_became_peered":"2024-09-17T09:26:20.777315+0000","last_unstale":"2024-09-17T09:26:47.027869+0000","last_undegraded":"2024-09-17T09:26:47.027869+0000","last_fullsized":"2024-09-17T09:26:47.027869+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:41.814566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474533+0000","last_change":"2024-09-17T09:26:16.745545+0000","last_active":"2024-09-17T09:26:46.474533+0000","last_peered":"2024-09-17T09:26:46.474533+0000","last_clean":"2024-09-17T09:26:46.474533+0000","last_became_active":"2024-09-17T09:26:16.745284+0000","last_became_peered":"2024-09-17T09:26:16.745284+0000","last_unstale":"2024-09-17T09:26:46.474533+0000","last_undegraded":"2024-09-17T09:26:46.474533+0000","last_fullsized":"2024-09-17T09:26:46.474533+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:55:25.499228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756620+0000","last_change":"2024-09-17T09:26:14.724081+0000","last_active":"2024-09-17T09:26:46.756620+0000","last_peered":"2024-09-17T09:26:46.756620+0000","last_clean":"2024-09-17T09:26:46.756620+0000","last_became_active":"2024-09-17T09:26:14.723767+0000","last_became_peered":"2024-09-17T09:26:14.723767+0000","last_unstale":"2024-09-17T09:26:46.756620+0000","last_undegraded":"2024-09-17T09:26:46.756620+0000","last_fullsized":"2024-09-17T09:26:46.756620+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:16.751196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'11","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:52.948433+0000","last_change":"2024-09-17T09:26:18.762481+0000","last_active":"2024-09-17T09:27:52.948433+0000","last_peered":"2024-09-17T09:27:52.948433+0000","last_clean":"2024-09-17T09:27:52.948433+0000","last_became_active":"2024-09-17T09:26:18.762206+0000","last_became_peered":"2024-09-17T09:26:18.762206+0000","last_unstale":"2024-09-17T09:27:52.948433+0000","last_undegraded":"2024-09-17T09:27:52.948433+0000","last_fullsized":"2024-09-17T09:27:52.948433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:48.844460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871396+0000","last_change":"2024-09-17T09:26:20.781202+0000","last_active":"2024-09-17T09:26:46.871396+0000","last_peered":"2024-09-17T09:26:46.871396+0000","last_clean":"2024-09-17T09:26:46.871396+0000","last_became_active":"2024-09-17T09:26:20.780480+0000","last_became_peered":"2024-09-17T09:26:20.780480+0000","last_unstale":"2024-09-17T09:26:46.871396+0000","last_undegraded":"2024-09-17T09:26:46.871396+0000","last_fullsized":"2024-09-17T09:26:46.871396+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:38.663263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474076+0000","last_change":"2024-09-17T09:26:16.747931+0000","last_active":"2024-09-17T09:26:46.474076+0000","last_peered":"2024-09-17T09:26:46.474076+0000","last_clean":"2024-09-17T09:26:46.474076+0000","last_became_active":"2024-09-17T09:26:16.747579+0000","last_became_peered":"2024-09-17T09:26:16.747579+0000","last_unstale":"2024-09-17T09:26:46.474076+0000","last_undegraded":"2024-09-17T09:26:46.474076+0000","last_fullsized":"2024-09-17T09:26:46.474076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:56:43.122018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756554+0000","last_change":"2024-09-17T09:26:14.725459+0000","last_active":"2024-09-17T09:26:46.756554+0000","last_peered":"2024-09-17T09:26:46.756554+0000","last_clean":"2024-09-17T09:26:46.756554+0000","last_became_active":"2024-09-17T09:26:14.725204+0000","last_became_peered":"2024-09-17T09:26:14.725204+0000","last_unstale":"2024-09-17T09:26:46.756554+0000","last_undegraded":"2024-09-17T09:26:46.756554+0000","last_fullsized":"2024-09-17T09:26:46.756554+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:45.348231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'11","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:52.948516+0000","last_change":"2024-09-17T09:26:18.764740+0000","last_active":"2024-09-17T09:27:52.948516+0000","last_peered":"2024-09-17T09:27:52.948516+0000","last_clean":"2024-09-17T09:27:52.948516+0000","last_became_active":"2024-09-17T09:26:18.764479+0000","last_became_peered":"2024-09-17T09:26:18.764479+0000","last_unstale":"2024-09-17T09:27:52.948516+0000","last_undegraded":"2024-09-17T09:27:52.948516+0000","last_fullsized":"2024-09-17T09:27:52.948516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:45.303237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476087+0000","last_change":"2024-09-17T09:26:20.775266+0000","last_active":"2024-09-17T09:26:46.476087+0000","last_peered":"2024-09-17T09:26:46.476087+0000","last_clean":"2024-09-17T09:26:46.476087+0000","last_became_active":"2024-09-17T09:26:20.774891+0000","last_became_peered":"2024-09-17T09:26:20.774891+0000","last_unstale":"2024-09-17T09:26:46.476087+0000","last_undegraded":"2024-09-17T09:26:46.476087+0000","last_fullsized":"2024-09-17T09:26:46.476087+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:25:05.846709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"67'12","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871191+0000","last_change":"2024-09-17T09:26:16.813501+0000","last_active":"2024-09-17T09:26:46.871191+0000","last_peered":"2024-09-17T09:26:46.871191+0000","last_clean":"2024-09-17T09:26:46.871191+0000","last_became_active":"2024-09-17T09:26:16.813196+0000","last_became_peered":"2024-09-17T09:26:16.813196+0000","last_unstale":"2024-09-17T09:26:46.871191+0000","last_undegraded":"2024-09-17T09:26:46.871191+0000","last_fullsized":"2024-09-17T09:26:46.871191+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:39.868980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473517+0000","last_change":"2024-09-17T09:26:14.728109+0000","last_active":"2024-09-17T09:26:46.473517+0000","last_peered":"2024-09-17T09:26:46.473517+0000","last_clean":"2024-09-17T09:26:46.473517+0000","last_became_active":"2024-09-17T09:26:14.727819+0000","last_became_peered":"2024-09-17T09:26:14.727819+0000","last_unstale":"2024-09-17T09:26:46.473517+0000","last_undegraded":"2024-09-17T09:26:46.473517+0000","last_fullsized":"2024-09-17T09:26:46.473517+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:02.267972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476360+0000","last_change":"2024-09-17T09:26:18.764783+0000","last_active":"2024-09-17T09:26:46.476360+0000","last_peered":"2024-09-17T09:26:46.476360+0000","last_clean":"2024-09-17T09:26:46.476360+0000","last_became_active":"2024-09-17T09:26:18.763820+0000","last_became_peered":"2024-09-17T09:26:18.763820+0000","last_unstale":"2024-09-17T09:26:46.476360+0000","last_undegraded":"2024-09-17T09:26:46.476360+0000","last_fullsized":"2024-09-17T09:26:46.476360+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:36:24.449240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474508+0000","last_change":"2024-09-17T09:26:21.247439+0000","last_active":"2024-09-17T09:26:46.474508+0000","last_peered":"2024-09-17T09:26:46.474508+0000","last_clean":"2024-09-17T09:26:46.474508+0000","last_became_active":"2024-09-17T09:26:21.247088+0000","last_became_peered":"2024-09-17T09:26:21.247088+0000","last_unstale":"2024-09-17T09:26:46.474508+0000","last_undegraded":"2024-09-17T09:26:46.474508+0000","last_fullsized":"2024-09-17T09:26:46.474508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:06.886592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"67'12","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273914+0000","last_change":"2024-09-17T09:26:16.743884+0000","last_active":"2024-09-17T09:26:47.273914+0000","last_peered":"2024-09-17T09:26:47.273914+0000","last_clean":"2024-09-17T09:26:47.273914+0000","last_became_active":"2024-09-17T09:26:16.743580+0000","last_became_peered":"2024-09-17T09:26:16.743580+0000","last_unstale":"2024-09-17T09:26:47.273914+0000","last_undegraded":"2024-09-17T09:26:47.273914+0000","last_fullsized":"2024-09-17T09:26:47.273914+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:00.691635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223329+0000","last_change":"2024-09-17T09:26:14.724391+0000","last_active":"2024-09-17T09:26:47.223329+0000","last_peered":"2024-09-17T09:26:47.223329+0000","last_clean":"2024-09-17T09:26:47.223329+0000","last_became_active":"2024-09-17T09:26:14.724196+0000","last_became_peered":"2024-09-17T09:26:14.724196+0000","last_unstale":"2024-09-17T09:26:47.223329+0000","last_undegraded":"2024-09-17T09:26:47.223329+0000","last_fullsized":"2024-09-17T09:26:47.223329+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:01.616900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'1","reported_seq":25,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871049+0000","last_change":"2024-09-17T09:26:20.760952+0000","last_active":"2024-09-17T09:26:46.871049+0000","last_peered":"2024-09-17T09:26:46.871049+0000","last_clean":"2024-09-17T09:26:46.871049+0000","last_became_active":"2024-09-17T09:26:16.745279+0000","last_became_peered":"2024-09-17T09:26:16.745279+0000","last_unstale":"2024-09-17T09:26:46.871049+0000","last_undegraded":"2024-09-17T09:26:46.871049+0000","last_fullsized":"2024-09-17T09:26:46.871049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:49:29.689044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000380288,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"67'11","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:52.948462+0000","last_change":"2024-09-17T09:26:18.760293+0000","last_active":"2024-09-17T09:27:52.948462+0000","last_peered":"2024-09-17T09:27:52.948462+0000","last_clean":"2024-09-17T09:27:52.948462+0000","last_became_active":"2024-09-17T09:26:18.759860+0000","last_became_peered":"2024-09-17T09:26:18.759860+0000","last_unstale":"2024-09-17T09:27:52.948462+0000","last_undegraded":"2024-09-17T09:27:52.948462+0000","last_fullsized":"2024-09-17T09:27:52.948462+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:22.903002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473615+0000","last_change":"2024-09-17T09:26:21.250808+0000","last_active":"2024-09-17T09:26:46.473615+0000","last_peered":"2024-09-17T09:26:46.473615+0000","last_clean":"2024-09-17T09:26:46.473615+0000","last_became_active":"2024-09-17T09:26:21.250520+0000","last_became_peered":"2024-09-17T09:26:21.250520+0000","last_unstale":"2024-09-17T09:26:46.473615+0000","last_undegraded":"2024-09-17T09:26:46.473615+0000","last_fullsized":"2024-09-17T09:26:46.473615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:02:00.601554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"67'13","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.475038+0000","last_change":"2024-09-17T09:26:16.748071+0000","last_active":"2024-09-17T09:26:46.475038+0000","last_peered":"2024-09-17T09:26:46.475038+0000","last_clean":"2024-09-17T09:26:46.475038+0000","last_became_active":"2024-09-17T09:26:16.747767+0000","last_became_peered":"2024-09-17T09:26:16.747767+0000","last_unstale":"2024-09-17T09:26:46.475038+0000","last_undegraded":"2024-09-17T09:26:46.475038+0000","last_fullsized":"2024-09-17T09:26:46.475038+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:17:33.397506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757657+0000","last_change":"2024-09-17T09:26:14.721028+0000","last_active":"2024-09-17T09:26:46.757657+0000","last_peered":"2024-09-17T09:26:46.757657+0000","last_clean":"2024-09-17T09:26:46.757657+0000","last_became_active":"2024-09-17T09:26:14.720662+0000","last_became_peered":"2024-09-17T09:26:14.720662+0000","last_unstale":"2024-09-17T09:26:46.757657+0000","last_undegraded":"2024-09-17T09:26:46.757657+0000","last_fullsized":"2024-09-17T09:26:46.757657+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:53.527988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"68'5","reported_seq":124,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:58.288268+0000","last_change":"2024-09-17T09:26:20.775567+0000","last_active":"2024-09-17T09:27:58.288268+0000","last_peered":"2024-09-17T09:27:58.288268+0000","last_clean":"2024-09-17T09:27:58.288268+0000","last_became_active":"2024-09-17T09:26:16.747710+0000","last_became_peered":"2024-09-17T09:26:16.747710+0000","last_unstale":"2024-09-17T09:27:58.288268+0000","last_undegraded":"2024-09-17T09:27:58.288268+0000","last_fullsized":"2024-09-17T09:27:58.288268+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:46.061292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028468,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":97,"num_read_kb":92,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.870868+0000","last_change":"2024-09-17T09:26:18.760591+0000","last_active":"2024-09-17T09:26:46.870868+0000","last_peered":"2024-09-17T09:26:46.870868+0000","last_clean":"2024-09-17T09:26:46.870868+0000","last_became_active":"2024-09-17T09:26:18.760318+0000","last_became_peered":"2024-09-17T09:26:18.760318+0000","last_unstale":"2024-09-17T09:26:46.870868+0000","last_undegraded":"2024-09-17T09:26:46.870868+0000","last_fullsized":"2024-09-17T09:26:46.870868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:46:37.291405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.870948+0000","last_change":"2024-09-17T09:26:20.777409+0000","last_active":"2024-09-17T09:26:46.870948+0000","last_peered":"2024-09-17T09:26:46.870948+0000","last_clean":"2024-09-17T09:26:46.870948+0000","last_became_active":"2024-09-17T09:26:20.777098+0000","last_became_peered":"2024-09-17T09:26:20.777098+0000","last_unstale":"2024-09-17T09:26:46.870948+0000","last_undegraded":"2024-09-17T09:26:46.870948+0000","last_fullsized":"2024-09-17T09:26:46.870948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:12.837968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"67'30","reported_seq":91,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:52.948433+0000","last_change":"2024-09-17T09:26:16.812376+0000","last_active":"2024-09-17T09:27:52.948433+0000","last_peered":"2024-09-17T09:27:52.948433+0000","last_clean":"2024-09-17T09:27:52.948433+0000","last_became_active":"2024-09-17T09:26:16.812057+0000","last_became_peered":"2024-09-17T09:26:16.812057+0000","last_unstale":"2024-09-17T09:27:52.948433+0000","last_undegraded":"2024-09-17T09:27:52.948433+0000","last_fullsized":"2024-09-17T09:27:52.948433+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:50.650693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474422+0000","last_change":"2024-09-17T09:26:14.732665+0000","last_active":"2024-09-17T09:26:46.474422+0000","last_peered":"2024-09-17T09:26:46.474422+0000","last_clean":"2024-09-17T09:26:46.474422+0000","last_became_active":"2024-09-17T09:26:14.732389+0000","last_became_peered":"2024-09-17T09:26:14.732389+0000","last_unstale":"2024-09-17T09:26:46.474422+0000","last_undegraded":"2024-09-17T09:26:46.474422+0000","last_fullsized":"2024-09-17T09:26:46.474422+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:11:46.814321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223749+0000","last_change":"2024-09-17T09:26:18.767936+0000","last_active":"2024-09-17T09:26:47.223749+0000","last_peered":"2024-09-17T09:26:47.223749+0000","last_clean":"2024-09-17T09:26:47.223749+0000","last_became_active":"2024-09-17T09:26:18.767650+0000","last_became_peered":"2024-09-17T09:26:18.767650+0000","last_unstale":"2024-09-17T09:26:47.223749+0000","last_undegraded":"2024-09-17T09:26:47.223749+0000","last_fullsized":"2024-09-17T09:26:47.223749+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:42.177315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756914+0000","last_change":"2024-09-17T09:26:21.248934+0000","last_active":"2024-09-17T09:26:46.756914+0000","last_peered":"2024-09-17T09:26:46.756914+0000","last_clean":"2024-09-17T09:26:46.756914+0000","last_became_active":"2024-09-17T09:26:21.248575+0000","last_became_peered":"2024-09-17T09:26:21.248575+0000","last_unstale":"2024-09-17T09:26:46.756914+0000","last_undegraded":"2024-09-17T09:26:46.756914+0000","last_fullsized":"2024-09-17T09:26:46.756914+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:39.167812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"67'14","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476301+0000","last_change":"2024-09-17T09:26:16.812435+0000","last_active":"2024-09-17T09:26:46.476301+0000","last_peered":"2024-09-17T09:26:46.476301+0000","last_clean":"2024-09-17T09:26:46.476301+0000","last_became_active":"2024-09-17T09:26:16.812159+0000","last_became_peered":"2024-09-17T09:26:16.812159+0000","last_unstale":"2024-09-17T09:26:46.476301+0000","last_undegraded":"2024-09-17T09:26:46.476301+0000","last_fullsized":"2024-09-17T09:26:46.476301+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:34:57.633416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756610+0000","last_change":"2024-09-17T09:26:14.728946+0000","last_active":"2024-09-17T09:26:46.756610+0000","last_peered":"2024-09-17T09:26:46.756610+0000","last_clean":"2024-09-17T09:26:46.756610+0000","last_became_active":"2024-09-17T09:26:14.728645+0000","last_became_peered":"2024-09-17T09:26:14.728645+0000","last_unstale":"2024-09-17T09:26:46.756610+0000","last_undegraded":"2024-09-17T09:26:46.756610+0000","last_fullsized":"2024-09-17T09:26:46.756610+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:53.731440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"66'2","reported_seq":26,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757295+0000","last_change":"2024-09-17T09:26:20.767626+0000","last_active":"2024-09-17T09:26:46.757295+0000","last_peered":"2024-09-17T09:26:46.757295+0000","last_clean":"2024-09-17T09:26:46.757295+0000","last_became_active":"2024-09-17T09:26:16.747334+0000","last_became_peered":"2024-09-17T09:26:16.747334+0000","last_unstale":"2024-09-17T09:26:46.757295+0000","last_undegraded":"2024-09-17T09:26:46.757295+0000","last_fullsized":"2024-09-17T09:26:46.757295+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.735859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013679499999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"67'11","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:52.948434+0000","last_change":"2024-09-17T09:26:18.765998+0000","last_active":"2024-09-17T09:27:52.948434+0000","last_peered":"2024-09-17T09:27:52.948434+0000","last_clean":"2024-09-17T09:27:52.948434+0000","last_became_active":"2024-09-17T09:26:18.765705+0000","last_became_peered":"2024-09-17T09:26:18.765705+0000","last_unstale":"2024-09-17T09:27:52.948434+0000","last_undegraded":"2024-09-17T09:27:52.948434+0000","last_fullsized":"2024-09-17T09:27:52.948434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:57:21.622471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273149+0000","last_change":"2024-09-17T09:26:20.780175+0000","last_active":"2024-09-17T09:26:47.273149+0000","last_peered":"2024-09-17T09:26:47.273149+0000","last_clean":"2024-09-17T09:26:47.273149+0000","last_became_active":"2024-09-17T09:26:20.779862+0000","last_became_peered":"2024-09-17T09:26:20.779862+0000","last_unstale":"2024-09-17T09:26:47.273149+0000","last_undegraded":"2024-09-17T09:26:47.273149+0000","last_fullsized":"2024-09-17T09:26:47.273149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:20:42.666024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"67'19","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871355+0000","last_change":"2024-09-17T09:26:16.744687+0000","last_active":"2024-09-17T09:26:46.871355+0000","last_peered":"2024-09-17T09:26:46.871355+0000","last_clean":"2024-09-17T09:26:46.871355+0000","last_became_active":"2024-09-17T09:26:16.744078+0000","last_became_peered":"2024-09-17T09:26:16.744078+0000","last_unstale":"2024-09-17T09:26:46.871355+0000","last_undegraded":"2024-09-17T09:26:46.871355+0000","last_fullsized":"2024-09-17T09:26:46.871355+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:11.409204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"60'1","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476083+0000","last_change":"2024-09-17T09:26:14.724555+0000","last_active":"2024-09-17T09:26:46.476083+0000","last_peered":"2024-09-17T09:26:46.476083+0000","last_clean":"2024-09-17T09:26:46.476083+0000","last_became_active":"2024-09-17T09:26:14.724253+0000","last_became_peered":"2024-09-17T09:26:14.724253+0000","last_unstale":"2024-09-17T09:26:46.476083+0000","last_undegraded":"2024-09-17T09:26:46.476083+0000","last_fullsized":"2024-09-17T09:26:46.476083+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:04:44.010016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273749+0000","last_change":"2024-09-17T09:26:18.765039+0000","last_active":"2024-09-17T09:26:47.273749+0000","last_peered":"2024-09-17T09:26:47.273749+0000","last_clean":"2024-09-17T09:26:47.273749+0000","last_became_active":"2024-09-17T09:26:18.764779+0000","last_became_peered":"2024-09-17T09:26:18.764779+0000","last_unstale":"2024-09-17T09:26:47.273749+0000","last_undegraded":"2024-09-17T09:26:47.273749+0000","last_fullsized":"2024-09-17T09:26:47.273749+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:17.139829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474808+0000","last_change":"2024-09-17T09:26:20.780934+0000","last_active":"2024-09-17T09:26:46.474808+0000","last_peered":"2024-09-17T09:26:46.474808+0000","last_clean":"2024-09-17T09:26:46.474808+0000","last_became_active":"2024-09-17T09:26:20.780602+0000","last_became_peered":"2024-09-17T09:26:20.780602+0000","last_unstale":"2024-09-17T09:26:46.474808+0000","last_undegraded":"2024-09-17T09:26:46.474808+0000","last_fullsized":"2024-09-17T09:26:46.474808+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:33.807274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"67'18","reported_seq":51,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757172+0000","last_change":"2024-09-17T09:26:16.813661+0000","last_active":"2024-09-17T09:26:46.757172+0000","last_peered":"2024-09-17T09:26:46.757172+0000","last_clean":"2024-09-17T09:26:46.757172+0000","last_became_active":"2024-09-17T09:26:16.813407+0000","last_became_peered":"2024-09-17T09:26:16.813407+0000","last_unstale":"2024-09-17T09:26:46.757172+0000","last_undegraded":"2024-09-17T09:26:46.757172+0000","last_fullsized":"2024-09-17T09:26:46.757172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:44.012466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027047+0000","last_change":"2024-09-17T09:26:14.731301+0000","last_active":"2024-09-17T09:26:47.027047+0000","last_peered":"2024-09-17T09:26:47.027047+0000","last_clean":"2024-09-17T09:26:47.027047+0000","last_became_active":"2024-09-17T09:26:14.731022+0000","last_became_peered":"2024-09-17T09:26:14.731022+0000","last_unstale":"2024-09-17T09:26:47.027047+0000","last_undegraded":"2024-09-17T09:26:47.027047+0000","last_fullsized":"2024-09-17T09:26:47.027047+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:42.821585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027807+0000","last_change":"2024-09-17T09:26:18.762625+0000","last_active":"2024-09-17T09:26:47.027807+0000","last_peered":"2024-09-17T09:26:47.027807+0000","last_clean":"2024-09-17T09:26:47.027807+0000","last_became_active":"2024-09-17T09:26:18.762319+0000","last_became_peered":"2024-09-17T09:26:18.762319+0000","last_unstale":"2024-09-17T09:26:47.027807+0000","last_undegraded":"2024-09-17T09:26:47.027807+0000","last_fullsized":"2024-09-17T09:26:47.027807+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:45:30.058597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474056+0000","last_change":"2024-09-17T09:26:21.247196+0000","last_active":"2024-09-17T09:26:46.474056+0000","last_peered":"2024-09-17T09:26:46.474056+0000","last_clean":"2024-09-17T09:26:46.474056+0000","last_became_active":"2024-09-17T09:26:21.246857+0000","last_became_peered":"2024-09-17T09:26:21.246857+0000","last_unstale":"2024-09-17T09:26:46.474056+0000","last_undegraded":"2024-09-17T09:26:46.474056+0000","last_fullsized":"2024-09-17T09:26:46.474056+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:39.740778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"67'14","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273289+0000","last_change":"2024-09-17T09:26:16.746320+0000","last_active":"2024-09-17T09:26:47.273289+0000","last_peered":"2024-09-17T09:26:47.273289+0000","last_clean":"2024-09-17T09:26:47.273289+0000","last_became_active":"2024-09-17T09:26:16.746043+0000","last_became_peered":"2024-09-17T09:26:16.746043+0000","last_unstale":"2024-09-17T09:26:47.273289+0000","last_undegraded":"2024-09-17T09:26:47.273289+0000","last_fullsized":"2024-09-17T09:26:47.273289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:00.208644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473535+0000","last_change":"2024-09-17T09:26:14.728452+0000","last_active":"2024-09-17T09:26:46.473535+0000","last_peered":"2024-09-17T09:26:46.473535+0000","last_clean":"2024-09-17T09:26:46.473535+0000","last_became_active":"2024-09-17T09:26:14.728170+0000","last_became_peered":"2024-09-17T09:26:14.728170+0000","last_unstale":"2024-09-17T09:26:46.473535+0000","last_undegraded":"2024-09-17T09:26:46.473535+0000","last_fullsized":"2024-09-17T09:26:46.473535+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:59.581480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476496+0000","last_change":"2024-09-17T09:26:18.764967+0000","last_active":"2024-09-17T09:26:46.476496+0000","last_peered":"2024-09-17T09:26:46.476496+0000","last_clean":"2024-09-17T09:26:46.476496+0000","last_became_active":"2024-09-17T09:26:18.764527+0000","last_became_peered":"2024-09-17T09:26:18.764527+0000","last_unstale":"2024-09-17T09:26:46.476496+0000","last_undegraded":"2024-09-17T09:26:46.476496+0000","last_fullsized":"2024-09-17T09:26:46.476496+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:56.753897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757220+0000","last_change":"2024-09-17T09:26:20.778571+0000","last_active":"2024-09-17T09:26:46.757220+0000","last_peered":"2024-09-17T09:26:46.757220+0000","last_clean":"2024-09-17T09:26:46.757220+0000","last_became_active":"2024-09-17T09:26:20.778254+0000","last_became_peered":"2024-09-17T09:26:20.778254+0000","last_unstale":"2024-09-17T09:26:46.757220+0000","last_undegraded":"2024-09-17T09:26:46.757220+0000","last_fullsized":"2024-09-17T09:26:46.757220+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:46:05.622258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474264+0000","last_change":"2024-09-17T09:26:16.742822+0000","last_active":"2024-09-17T09:26:46.474264+0000","last_peered":"2024-09-17T09:26:46.474264+0000","last_clean":"2024-09-17T09:26:46.474264+0000","last_became_active":"2024-09-17T09:26:16.742546+0000","last_became_peered":"2024-09-17T09:26:16.742546+0000","last_unstale":"2024-09-17T09:26:46.474264+0000","last_undegraded":"2024-09-17T09:26:46.474264+0000","last_fullsized":"2024-09-17T09:26:46.474264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:08:07.193982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.029055+0000","last_change":"2024-09-17T09:26:47.028633+0000","last_active":"2024-09-17T09:26:47.029055+0000","last_peered":"2024-09-17T09:26:47.029055+0000","last_clean":"2024-09-17T09:26:47.029055+0000","last_became_active":"2024-09-17T09:26:47.028221+0000","last_became_peered":"2024-09-17T09:26:47.028221+0000","last_unstale":"2024-09-17T09:26:47.029055+0000","last_undegraded":"2024-09-17T09:26:47.029055+0000","last_fullsized":"2024-09-17T09:26:47.029055+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:00.419613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'188","reported_seq":336,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:00.590292+0000","last_change":"2024-09-17T09:26:05.194885+0000","last_active":"2024-09-17T09:27:00.590292+0000","last_peered":"2024-09-17T09:27:00.590292+0000","last_clean":"2024-09-17T09:27:00.590292+0000","last_became_active":"2024-09-17T09:26:05.194420+0000","last_became_peered":"2024-09-17T09:26:05.194420+0000","last_unstale":"2024-09-17T09:27:00.590292+0000","last_undegraded":"2024-09-17T09:27:00.590292+0000","last_fullsized":"2024-09-17T09:27:00.590292+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:23:58.802554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:23:58.802554+0000","last_clean_scrub_stamp":"2024-09-17T09:23:58.802554+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:08.801262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473926+0000","last_change":"2024-09-17T09:26:18.767051+0000","last_active":"2024-09-17T09:26:46.473926+0000","last_peered":"2024-09-17T09:26:46.473926+0000","last_clean":"2024-09-17T09:26:46.473926+0000","last_became_active":"2024-09-17T09:26:18.766746+0000","last_became_peered":"2024-09-17T09:26:18.766746+0000","last_unstale":"2024-09-17T09:26:46.473926+0000","last_undegraded":"2024-09-17T09:26:46.473926+0000","last_fullsized":"2024-09-17T09:26:46.473926+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:23.507097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476388+0000","last_change":"2024-09-17T09:26:20.777416+0000","last_active":"2024-09-17T09:26:46.476388+0000","last_peered":"2024-09-17T09:26:46.476388+0000","last_clean":"2024-09-17T09:26:46.476388+0000","last_became_active":"2024-09-17T09:26:20.777096+0000","last_became_peered":"2024-09-17T09:26:20.777096+0000","last_unstale":"2024-09-17T09:26:46.476388+0000","last_undegraded":"2024-09-17T09:26:46.476388+0000","last_fullsized":"2024-09-17T09:26:46.476388+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:09.613262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"67'17","reported_seq":47,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474347+0000","last_change":"2024-09-17T09:26:16.741985+0000","last_active":"2024-09-17T09:26:46.474347+0000","last_peered":"2024-09-17T09:26:46.474347+0000","last_clean":"2024-09-17T09:26:46.474347+0000","last_became_active":"2024-09-17T09:26:16.741689+0000","last_became_peered":"2024-09-17T09:26:16.741689+0000","last_unstale":"2024-09-17T09:26:46.474347+0000","last_undegraded":"2024-09-17T09:26:46.474347+0000","last_fullsized":"2024-09-17T09:26:46.474347+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:55.034586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027126+0000","last_change":"2024-09-17T09:26:14.729373+0000","last_active":"2024-09-17T09:26:47.027126+0000","last_peered":"2024-09-17T09:26:47.027126+0000","last_clean":"2024-09-17T09:26:47.027126+0000","last_became_active":"2024-09-17T09:26:14.729081+0000","last_became_peered":"2024-09-17T09:26:14.729081+0000","last_unstale":"2024-09-17T09:26:47.027126+0000","last_undegraded":"2024-09-17T09:26:47.027126+0000","last_fullsized":"2024-09-17T09:26:47.027126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:46.245436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027583+0000","last_change":"2024-09-17T09:26:18.760963+0000","last_active":"2024-09-17T09:26:47.027583+0000","last_peered":"2024-09-17T09:26:47.027583+0000","last_clean":"2024-09-17T09:26:47.027583+0000","last_became_active":"2024-09-17T09:26:18.760655+0000","last_became_peered":"2024-09-17T09:26:18.760655+0000","last_unstale":"2024-09-17T09:26:47.027583+0000","last_undegraded":"2024-09-17T09:26:47.027583+0000","last_fullsized":"2024-09-17T09:26:47.027583+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:11.885213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473655+0000","last_change":"2024-09-17T09:26:20.780780+0000","last_active":"2024-09-17T09:26:46.473655+0000","last_peered":"2024-09-17T09:26:46.473655+0000","last_clean":"2024-09-17T09:26:46.473655+0000","last_became_active":"2024-09-17T09:26:20.780525+0000","last_became_peered":"2024-09-17T09:26:20.780525+0000","last_unstale":"2024-09-17T09:26:46.473655+0000","last_undegraded":"2024-09-17T09:26:46.473655+0000","last_fullsized":"2024-09-17T09:26:46.473655+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:30.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476322+0000","last_change":"2024-09-17T09:26:16.743425+0000","last_active":"2024-09-17T09:26:46.476322+0000","last_peered":"2024-09-17T09:26:46.476322+0000","last_clean":"2024-09-17T09:26:46.476322+0000","last_became_active":"2024-09-17T09:26:16.743182+0000","last_became_peered":"2024-09-17T09:26:16.743182+0000","last_unstale":"2024-09-17T09:26:46.476322+0000","last_undegraded":"2024-09-17T09:26:46.476322+0000","last_fullsized":"2024-09-17T09:26:46.476322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:56.606826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756662+0000","last_change":"2024-09-17T09:26:14.724679+0000","last_active":"2024-09-17T09:26:46.756662+0000","last_peered":"2024-09-17T09:26:46.756662+0000","last_clean":"2024-09-17T09:26:46.756662+0000","last_became_active":"2024-09-17T09:26:14.724432+0000","last_became_peered":"2024-09-17T09:26:14.724432+0000","last_unstale":"2024-09-17T09:26:46.756662+0000","last_undegraded":"2024-09-17T09:26:46.756662+0000","last_fullsized":"2024-09-17T09:26:46.756662+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:36:47.957658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027485+0000","last_change":"2024-09-17T09:26:18.760179+0000","last_active":"2024-09-17T09:26:47.027485+0000","last_peered":"2024-09-17T09:26:47.027485+0000","last_clean":"2024-09-17T09:26:47.027485+0000","last_became_active":"2024-09-17T09:26:18.759842+0000","last_became_peered":"2024-09-17T09:26:18.759842+0000","last_unstale":"2024-09-17T09:26:47.027485+0000","last_undegraded":"2024-09-17T09:26:47.027485+0000","last_fullsized":"2024-09-17T09:26:47.027485+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:24:29.915445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273395+0000","last_change":"2024-09-17T09:26:20.779805+0000","last_active":"2024-09-17T09:26:47.273395+0000","last_peered":"2024-09-17T09:26:47.273395+0000","last_clean":"2024-09-17T09:26:47.273395+0000","last_became_active":"2024-09-17T09:26:20.779489+0000","last_became_peered":"2024-09-17T09:26:20.779489+0000","last_unstale":"2024-09-17T09:26:47.273395+0000","last_undegraded":"2024-09-17T09:26:47.273395+0000","last_fullsized":"2024-09-17T09:26:47.273395+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:33.662456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474114+0000","last_change":"2024-09-17T09:26:16.744297+0000","last_active":"2024-09-17T09:26:46.474114+0000","last_peered":"2024-09-17T09:26:46.474114+0000","last_clean":"2024-09-17T09:26:46.474114+0000","last_became_active":"2024-09-17T09:26:16.744030+0000","last_became_peered":"2024-09-17T09:26:16.744030+0000","last_unstale":"2024-09-17T09:26:46.474114+0000","last_undegraded":"2024-09-17T09:26:46.474114+0000","last_fullsized":"2024-09-17T09:26:46.474114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:41.597190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027125+0000","last_change":"2024-09-17T09:26:14.723780+0000","last_active":"2024-09-17T09:26:47.027125+0000","last_peered":"2024-09-17T09:26:47.027125+0000","last_clean":"2024-09-17T09:26:47.027125+0000","last_became_active":"2024-09-17T09:26:14.723501+0000","last_became_peered":"2024-09-17T09:26:14.723501+0000","last_unstale":"2024-09-17T09:26:47.027125+0000","last_undegraded":"2024-09-17T09:26:47.027125+0000","last_fullsized":"2024-09-17T09:26:47.027125+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:34.372643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"67'11","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:52.948552+0000","last_change":"2024-09-17T09:26:18.764303+0000","last_active":"2024-09-17T09:27:52.948552+0000","last_peered":"2024-09-17T09:27:52.948552+0000","last_clean":"2024-09-17T09:27:52.948552+0000","last_became_active":"2024-09-17T09:26:18.764024+0000","last_became_peered":"2024-09-17T09:26:18.764024+0000","last_unstale":"2024-09-17T09:27:52.948552+0000","last_undegraded":"2024-09-17T09:27:52.948552+0000","last_fullsized":"2024-09-17T09:27:52.948552+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:40:59.047020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.475898+0000","last_change":"2024-09-17T09:26:21.247482+0000","last_active":"2024-09-17T09:26:46.475898+0000","last_peered":"2024-09-17T09:26:46.475898+0000","last_clean":"2024-09-17T09:26:46.475898+0000","last_became_active":"2024-09-17T09:26:21.247125+0000","last_became_peered":"2024-09-17T09:26:21.247125+0000","last_unstale":"2024-09-17T09:26:46.475898+0000","last_undegraded":"2024-09-17T09:26:46.475898+0000","last_fullsized":"2024-09-17T09:26:46.475898+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:58.759643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474391+0000","last_change":"2024-09-17T09:26:16.742841+0000","last_active":"2024-09-17T09:26:46.474391+0000","last_peered":"2024-09-17T09:26:46.474391+0000","last_clean":"2024-09-17T09:26:46.474391+0000","last_became_active":"2024-09-17T09:26:16.742582+0000","last_became_peered":"2024-09-17T09:26:16.742582+0000","last_unstale":"2024-09-17T09:26:46.474391+0000","last_undegraded":"2024-09-17T09:26:46.474391+0000","last_fullsized":"2024-09-17T09:26:46.474391+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:38:28.233133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.870322+0000","last_change":"2024-09-17T09:26:14.724221+0000","last_active":"2024-09-17T09:26:46.870322+0000","last_peered":"2024-09-17T09:26:46.870322+0000","last_clean":"2024-09-17T09:26:46.870322+0000","last_became_active":"2024-09-17T09:26:14.723921+0000","last_became_peered":"2024-09-17T09:26:14.723921+0000","last_unstale":"2024-09-17T09:26:46.870322+0000","last_undegraded":"2024-09-17T09:26:46.870322+0000","last_fullsized":"2024-09-17T09:26:46.870322+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:27.820473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027330+0000","last_change":"2024-09-17T09:26:18.762212+0000","last_active":"2024-09-17T09:26:47.027330+0000","last_peered":"2024-09-17T09:26:47.027330+0000","last_clean":"2024-09-17T09:26:47.027330+0000","last_became_active":"2024-09-17T09:26:18.761953+0000","last_became_peered":"2024-09-17T09:26:18.761953+0000","last_unstale":"2024-09-17T09:26:47.027330+0000","last_undegraded":"2024-09-17T09:26:47.027330+0000","last_fullsized":"2024-09-17T09:26:47.027330+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:51:46.020357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474327+0000","last_change":"2024-09-17T09:26:20.781461+0000","last_active":"2024-09-17T09:26:46.474327+0000","last_peered":"2024-09-17T09:26:46.474327+0000","last_clean":"2024-09-17T09:26:46.474327+0000","last_became_active":"2024-09-17T09:26:20.781087+0000","last_became_peered":"2024-09-17T09:26:20.781087+0000","last_unstale":"2024-09-17T09:26:46.474327+0000","last_undegraded":"2024-09-17T09:26:46.474327+0000","last_fullsized":"2024-09-17T09:26:46.474327+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:11.321299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473635+0000","last_change":"2024-09-17T09:26:16.746278+0000","last_active":"2024-09-17T09:26:46.473635+0000","last_peered":"2024-09-17T09:26:46.473635+0000","last_clean":"2024-09-17T09:26:46.473635+0000","last_became_active":"2024-09-17T09:26:16.744810+0000","last_became_peered":"2024-09-17T09:26:16.744810+0000","last_unstale":"2024-09-17T09:26:46.473635+0000","last_undegraded":"2024-09-17T09:26:46.473635+0000","last_fullsized":"2024-09-17T09:26:46.473635+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:20:35.063630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027450+0000","last_change":"2024-09-17T09:26:14.729692+0000","last_active":"2024-09-17T09:26:47.027450+0000","last_peered":"2024-09-17T09:26:47.027450+0000","last_clean":"2024-09-17T09:26:47.027450+0000","last_became_active":"2024-09-17T09:26:14.728873+0000","last_became_peered":"2024-09-17T09:26:14.728873+0000","last_unstale":"2024-09-17T09:26:47.027450+0000","last_undegraded":"2024-09-17T09:26:47.027450+0000","last_fullsized":"2024-09-17T09:26:47.027450+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:27:58.498470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474992+0000","last_change":"2024-09-17T09:26:18.764593+0000","last_active":"2024-09-17T09:26:46.474992+0000","last_peered":"2024-09-17T09:26:46.474992+0000","last_clean":"2024-09-17T09:26:46.474992+0000","last_became_active":"2024-09-17T09:26:18.764115+0000","last_became_peered":"2024-09-17T09:26:18.764115+0000","last_unstale":"2024-09-17T09:26:46.474992+0000","last_undegraded":"2024-09-17T09:26:46.474992+0000","last_fullsized":"2024-09-17T09:26:46.474992+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:56:07.366252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"67'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.027574+0000","last_change":"2024-09-17T09:26:20.773479+0000","last_active":"2024-09-17T09:26:47.027574+0000","last_peered":"2024-09-17T09:26:47.027574+0000","last_clean":"2024-09-17T09:26:47.027574+0000","last_became_active":"2024-09-17T09:26:20.772814+0000","last_became_peered":"2024-09-17T09:26:20.772814+0000","last_unstale":"2024-09-17T09:26:47.027574+0000","last_undegraded":"2024-09-17T09:26:47.027574+0000","last_fullsized":"2024-09-17T09:26:47.027574+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:34:32.784044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"67'4","reported_seq":25,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223569+0000","last_change":"2024-09-17T09:26:16.747139+0000","last_active":"2024-09-17T09:26:47.223569+0000","last_peered":"2024-09-17T09:26:47.223569+0000","last_clean":"2024-09-17T09:26:47.223569+0000","last_became_active":"2024-09-17T09:26:16.746854+0000","last_became_peered":"2024-09-17T09:26:16.746854+0000","last_unstale":"2024-09-17T09:26:47.223569+0000","last_undegraded":"2024-09-17T09:26:47.223569+0000","last_fullsized":"2024-09-17T09:26:47.223569+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:37:45.212652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.224025+0000","last_change":"2024-09-17T09:26:14.720744+0000","last_active":"2024-09-17T09:26:47.224025+0000","last_peered":"2024-09-17T09:26:47.224025+0000","last_clean":"2024-09-17T09:26:47.224025+0000","last_became_active":"2024-09-17T09:26:14.720014+0000","last_became_peered":"2024-09-17T09:26:14.720014+0000","last_unstale":"2024-09-17T09:26:47.224025+0000","last_undegraded":"2024-09-17T09:26:47.224025+0000","last_fullsized":"2024-09-17T09:26:47.224025+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:45:19.672879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476333+0000","last_change":"2024-09-17T09:26:18.762346+0000","last_active":"2024-09-17T09:26:46.476333+0000","last_peered":"2024-09-17T09:26:46.476333+0000","last_clean":"2024-09-17T09:26:46.476333+0000","last_became_active":"2024-09-17T09:26:18.762019+0000","last_became_peered":"2024-09-17T09:26:18.762019+0000","last_unstale":"2024-09-17T09:26:46.476333+0000","last_undegraded":"2024-09-17T09:26:46.476333+0000","last_fullsized":"2024-09-17T09:26:46.476333+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:47.565382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473968+0000","last_change":"2024-09-17T09:26:21.250107+0000","last_active":"2024-09-17T09:26:46.473968+0000","last_peered":"2024-09-17T09:26:46.473968+0000","last_clean":"2024-09-17T09:26:46.473968+0000","last_became_active":"2024-09-17T09:26:21.249800+0000","last_became_peered":"2024-09-17T09:26:21.249800+0000","last_unstale":"2024-09-17T09:26:46.473968+0000","last_undegraded":"2024-09-17T09:26:46.473968+0000","last_fullsized":"2024-09-17T09:26:46.473968+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:58:08.827879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"67'11","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473881+0000","last_change":"2024-09-17T09:26:16.812511+0000","last_active":"2024-09-17T09:26:46.473881+0000","last_peered":"2024-09-17T09:26:46.473881+0000","last_clean":"2024-09-17T09:26:46.473881+0000","last_became_active":"2024-09-17T09:26:16.812239+0000","last_became_peered":"2024-09-17T09:26:16.812239+0000","last_unstale":"2024-09-17T09:26:46.473881+0000","last_undegraded":"2024-09-17T09:26:46.473881+0000","last_fullsized":"2024-09-17T09:26:46.473881+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:12.466728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476013+0000","last_change":"2024-09-17T09:26:14.723105+0000","last_active":"2024-09-17T09:26:46.476013+0000","last_peered":"2024-09-17T09:26:46.476013+0000","last_clean":"2024-09-17T09:26:46.476013+0000","last_became_active":"2024-09-17T09:26:14.722780+0000","last_became_peered":"2024-09-17T09:26:14.722780+0000","last_unstale":"2024-09-17T09:26:46.476013+0000","last_undegraded":"2024-09-17T09:26:46.476013+0000","last_fullsized":"2024-09-17T09:26:46.476013+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.414459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'11","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:52.948541+0000","last_change":"2024-09-17T09:26:18.765530+0000","last_active":"2024-09-17T09:27:52.948541+0000","last_peered":"2024-09-17T09:27:52.948541+0000","last_clean":"2024-09-17T09:27:52.948541+0000","last_became_active":"2024-09-17T09:26:18.765271+0000","last_became_peered":"2024-09-17T09:26:18.765271+0000","last_unstale":"2024-09-17T09:27:52.948541+0000","last_undegraded":"2024-09-17T09:27:52.948541+0000","last_fullsized":"2024-09-17T09:27:52.948541+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:21.154061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273795+0000","last_change":"2024-09-17T09:26:20.780262+0000","last_active":"2024-09-17T09:26:47.273795+0000","last_peered":"2024-09-17T09:26:47.273795+0000","last_clean":"2024-09-17T09:26:47.273795+0000","last_became_active":"2024-09-17T09:26:20.779910+0000","last_became_peered":"2024-09-17T09:26:20.779910+0000","last_unstale":"2024-09-17T09:26:47.273795+0000","last_undegraded":"2024-09-17T09:26:47.273795+0000","last_fullsized":"2024-09-17T09:26:47.273795+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:27.752379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273397+0000","last_change":"2024-09-17T09:26:16.744738+0000","last_active":"2024-09-17T09:26:47.273397+0000","last_peered":"2024-09-17T09:26:47.273397+0000","last_clean":"2024-09-17T09:26:47.273397+0000","last_became_active":"2024-09-17T09:26:16.744456+0000","last_became_peered":"2024-09-17T09:26:16.744456+0000","last_unstale":"2024-09-17T09:26:47.273397+0000","last_undegraded":"2024-09-17T09:26:47.273397+0000","last_fullsized":"2024-09-17T09:26:47.273397+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:03.571673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.272774+0000","last_change":"2024-09-17T09:26:14.720843+0000","last_active":"2024-09-17T09:26:47.272774+0000","last_peered":"2024-09-17T09:26:47.272774+0000","last_clean":"2024-09-17T09:26:47.272774+0000","last_became_active":"2024-09-17T09:26:14.720564+0000","last_became_peered":"2024-09-17T09:26:14.720564+0000","last_unstale":"2024-09-17T09:26:47.272774+0000","last_undegraded":"2024-09-17T09:26:47.272774+0000","last_fullsized":"2024-09-17T09:26:47.272774+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:01.314290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'11","reported_seq":49,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:52.948490+0000","last_change":"2024-09-17T09:26:18.760606+0000","last_active":"2024-09-17T09:27:52.948490+0000","last_peered":"2024-09-17T09:27:52.948490+0000","last_clean":"2024-09-17T09:27:52.948490+0000","last_became_active":"2024-09-17T09:26:18.760123+0000","last_became_peered":"2024-09-17T09:26:18.760123+0000","last_unstale":"2024-09-17T09:27:52.948490+0000","last_undegraded":"2024-09-17T09:27:52.948490+0000","last_fullsized":"2024-09-17T09:27:52.948490+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:42.478141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871716+0000","last_change":"2024-09-17T09:26:20.780844+0000","last_active":"2024-09-17T09:26:46.871716+0000","last_peered":"2024-09-17T09:26:46.871716+0000","last_clean":"2024-09-17T09:26:46.871716+0000","last_became_active":"2024-09-17T09:26:20.780542+0000","last_became_peered":"2024-09-17T09:26:20.780542+0000","last_unstale":"2024-09-17T09:26:46.871716+0000","last_undegraded":"2024-09-17T09:26:46.871716+0000","last_fullsized":"2024-09-17T09:26:46.871716+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:29.566402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474301+0000","last_change":"2024-09-17T09:26:16.743577+0000","last_active":"2024-09-17T09:26:46.474301+0000","last_peered":"2024-09-17T09:26:46.474301+0000","last_clean":"2024-09-17T09:26:46.474301+0000","last_became_active":"2024-09-17T09:26:16.743284+0000","last_became_peered":"2024-09-17T09:26:16.743284+0000","last_unstale":"2024-09-17T09:26:46.474301+0000","last_undegraded":"2024-09-17T09:26:46.474301+0000","last_fullsized":"2024-09-17T09:26:46.474301+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:25.350577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223270+0000","last_change":"2024-09-17T09:26:14.726094+0000","last_active":"2024-09-17T09:26:47.223270+0000","last_peered":"2024-09-17T09:26:47.223270+0000","last_clean":"2024-09-17T09:26:47.223270+0000","last_became_active":"2024-09-17T09:26:14.725879+0000","last_became_peered":"2024-09-17T09:26:14.725879+0000","last_unstale":"2024-09-17T09:26:47.223270+0000","last_undegraded":"2024-09-17T09:26:47.223270+0000","last_fullsized":"2024-09-17T09:26:47.223270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:52.795278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474424+0000","last_change":"2024-09-17T09:26:18.760740+0000","last_active":"2024-09-17T09:26:46.474424+0000","last_peered":"2024-09-17T09:26:46.474424+0000","last_clean":"2024-09-17T09:26:46.474424+0000","last_became_active":"2024-09-17T09:26:18.760436+0000","last_became_peered":"2024-09-17T09:26:18.760436+0000","last_unstale":"2024-09-17T09:26:46.474424+0000","last_undegraded":"2024-09-17T09:26:46.474424+0000","last_fullsized":"2024-09-17T09:26:46.474424+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:17:18.481597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.272994+0000","last_change":"2024-09-17T09:26:20.775652+0000","last_active":"2024-09-17T09:26:47.272994+0000","last_peered":"2024-09-17T09:26:47.272994+0000","last_clean":"2024-09-17T09:26:47.272994+0000","last_became_active":"2024-09-17T09:26:20.775088+0000","last_became_peered":"2024-09-17T09:26:20.775088+0000","last_unstale":"2024-09-17T09:26:47.272994+0000","last_undegraded":"2024-09-17T09:26:47.272994+0000","last_fullsized":"2024-09-17T09:26:47.272994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:03.697272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"67'10","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871376+0000","last_change":"2024-09-17T09:26:16.748842+0000","last_active":"2024-09-17T09:26:46.871376+0000","last_peered":"2024-09-17T09:26:46.871376+0000","last_clean":"2024-09-17T09:26:46.871376+0000","last_became_active":"2024-09-17T09:26:16.748458+0000","last_became_peered":"2024-09-17T09:26:16.748458+0000","last_unstale":"2024-09-17T09:26:46.871376+0000","last_undegraded":"2024-09-17T09:26:46.871376+0000","last_fullsized":"2024-09-17T09:26:46.871376+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:21.515972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.756731+0000","last_change":"2024-09-17T09:26:14.728073+0000","last_active":"2024-09-17T09:26:46.756731+0000","last_peered":"2024-09-17T09:26:46.756731+0000","last_clean":"2024-09-17T09:26:46.756731+0000","last_became_active":"2024-09-17T09:26:14.727752+0000","last_became_peered":"2024-09-17T09:26:14.727752+0000","last_unstale":"2024-09-17T09:26:46.756731+0000","last_undegraded":"2024-09-17T09:26:46.756731+0000","last_fullsized":"2024-09-17T09:26:46.756731+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:56.351066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757103+0000","last_change":"2024-09-17T09:26:18.766856+0000","last_active":"2024-09-17T09:26:46.757103+0000","last_peered":"2024-09-17T09:26:46.757103+0000","last_clean":"2024-09-17T09:26:46.757103+0000","last_became_active":"2024-09-17T09:26:18.766558+0000","last_became_peered":"2024-09-17T09:26:18.766558+0000","last_unstale":"2024-09-17T09:26:46.757103+0000","last_undegraded":"2024-09-17T09:26:46.757103+0000","last_fullsized":"2024-09-17T09:26:46.757103+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:24.514423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473939+0000","last_change":"2024-09-17T09:26:20.783056+0000","last_active":"2024-09-17T09:26:46.473939+0000","last_peered":"2024-09-17T09:26:46.473939+0000","last_clean":"2024-09-17T09:26:46.473939+0000","last_became_active":"2024-09-17T09:26:20.782769+0000","last_became_peered":"2024-09-17T09:26:20.782769+0000","last_unstale":"2024-09-17T09:26:46.473939+0000","last_undegraded":"2024-09-17T09:26:46.473939+0000","last_fullsized":"2024-09-17T09:26:46.473939+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:48:26.748095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"67'6","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273979+0000","last_change":"2024-09-17T09:26:16.742808+0000","last_active":"2024-09-17T09:26:47.273979+0000","last_peered":"2024-09-17T09:26:47.273979+0000","last_clean":"2024-09-17T09:26:47.273979+0000","last_became_active":"2024-09-17T09:26:16.742540+0000","last_became_peered":"2024-09-17T09:26:16.742540+0000","last_unstale":"2024-09-17T09:26:47.273979+0000","last_undegraded":"2024-09-17T09:26:47.273979+0000","last_fullsized":"2024-09-17T09:26:47.273979+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:06:21.273210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"60'1","reported_seq":26,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476442+0000","last_change":"2024-09-17T09:26:14.720259+0000","last_active":"2024-09-17T09:26:46.476442+0000","last_peered":"2024-09-17T09:26:46.476442+0000","last_clean":"2024-09-17T09:26:46.476442+0000","last_became_active":"2024-09-17T09:26:14.719991+0000","last_became_peered":"2024-09-17T09:26:14.719991+0000","last_unstale":"2024-09-17T09:26:46.476442+0000","last_undegraded":"2024-09-17T09:26:46.476442+0000","last_fullsized":"2024-09-17T09:26:46.476442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:21.002411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223891+0000","last_change":"2024-09-17T09:26:18.767543+0000","last_active":"2024-09-17T09:26:47.223891+0000","last_peered":"2024-09-17T09:26:47.223891+0000","last_clean":"2024-09-17T09:26:47.223891+0000","last_became_active":"2024-09-17T09:26:18.767244+0000","last_became_peered":"2024-09-17T09:26:18.767244+0000","last_unstale":"2024-09-17T09:26:47.223891+0000","last_undegraded":"2024-09-17T09:26:47.223891+0000","last_fullsized":"2024-09-17T09:26:47.223891+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:42:38.686983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473470+0000","last_change":"2024-09-17T09:26:20.774288+0000","last_active":"2024-09-17T09:26:46.473470+0000","last_peered":"2024-09-17T09:26:46.473470+0000","last_clean":"2024-09-17T09:26:46.473470+0000","last_became_active":"2024-09-17T09:26:20.773915+0000","last_became_peered":"2024-09-17T09:26:20.773915+0000","last_unstale":"2024-09-17T09:26:46.473470+0000","last_undegraded":"2024-09-17T09:26:46.473470+0000","last_fullsized":"2024-09-17T09:26:46.473470+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:45.049464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.476358+0000","last_change":"2024-09-17T09:26:16.743528+0000","last_active":"2024-09-17T09:26:46.476358+0000","last_peered":"2024-09-17T09:26:46.476358+0000","last_clean":"2024-09-17T09:26:46.476358+0000","last_became_active":"2024-09-17T09:26:16.743065+0000","last_became_peered":"2024-09-17T09:26:16.743065+0000","last_unstale":"2024-09-17T09:26:46.476358+0000","last_undegraded":"2024-09-17T09:26:46.476358+0000","last_fullsized":"2024-09-17T09:26:46.476358+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:02:33.861468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223520+0000","last_change":"2024-09-17T09:26:14.725546+0000","last_active":"2024-09-17T09:26:47.223520+0000","last_peered":"2024-09-17T09:26:47.223520+0000","last_clean":"2024-09-17T09:26:47.223520+0000","last_became_active":"2024-09-17T09:26:14.725285+0000","last_became_peered":"2024-09-17T09:26:14.725285+0000","last_unstale":"2024-09-17T09:26:47.223520+0000","last_undegraded":"2024-09-17T09:26:47.223520+0000","last_fullsized":"2024-09-17T09:26:47.223520+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:28.057127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473325+0000","last_change":"2024-09-17T09:26:18.765072+0000","last_active":"2024-09-17T09:26:46.473325+0000","last_peered":"2024-09-17T09:26:46.473325+0000","last_clean":"2024-09-17T09:26:46.473325+0000","last_became_active":"2024-09-17T09:26:18.764824+0000","last_became_peered":"2024-09-17T09:26:18.764824+0000","last_unstale":"2024-09-17T09:26:46.473325+0000","last_undegraded":"2024-09-17T09:26:46.473325+0000","last_fullsized":"2024-09-17T09:26:46.473325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:17:17.825245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474516+0000","last_change":"2024-09-17T09:26:21.248366+0000","last_active":"2024-09-17T09:26:46.474516+0000","last_peered":"2024-09-17T09:26:46.474516+0000","last_clean":"2024-09-17T09:26:46.474516+0000","last_became_active":"2024-09-17T09:26:21.247942+0000","last_became_peered":"2024-09-17T09:26:21.247942+0000","last_unstale":"2024-09-17T09:26:46.474516+0000","last_undegraded":"2024-09-17T09:26:46.474516+0000","last_fullsized":"2024-09-17T09:26:46.474516+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:52.064235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473876+0000","last_change":"2024-09-17T09:26:20.781221+0000","last_active":"2024-09-17T09:26:46.473876+0000","last_peered":"2024-09-17T09:26:46.473876+0000","last_clean":"2024-09-17T09:26:46.473876+0000","last_became_active":"2024-09-17T09:26:20.780549+0000","last_became_peered":"2024-09-17T09:26:20.780549+0000","last_unstale":"2024-09-17T09:26:46.473876+0000","last_undegraded":"2024-09-17T09:26:46.473876+0000","last_fullsized":"2024-09-17T09:26:46.473876+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:11.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"67'15","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757016+0000","last_change":"2024-09-17T09:26:16.747672+0000","last_active":"2024-09-17T09:26:46.757016+0000","last_peered":"2024-09-17T09:26:46.757016+0000","last_clean":"2024-09-17T09:26:46.757016+0000","last_became_active":"2024-09-17T09:26:16.747379+0000","last_became_peered":"2024-09-17T09:26:16.747379+0000","last_unstale":"2024-09-17T09:26:46.757016+0000","last_undegraded":"2024-09-17T09:26:46.757016+0000","last_fullsized":"2024-09-17T09:26:46.757016+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:18.371366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.475608+0000","last_change":"2024-09-17T09:26:14.728190+0000","last_active":"2024-09-17T09:26:46.475608+0000","last_peered":"2024-09-17T09:26:46.475608+0000","last_clean":"2024-09-17T09:26:46.475608+0000","last_became_active":"2024-09-17T09:26:14.727874+0000","last_became_peered":"2024-09-17T09:26:14.727874+0000","last_unstale":"2024-09-17T09:26:46.475608+0000","last_undegraded":"2024-09-17T09:26:46.475608+0000","last_fullsized":"2024-09-17T09:26:46.475608+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:16:11.223541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"67'11","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T09:27:52.948579+0000","last_change":"2024-09-17T09:26:18.761105+0000","last_active":"2024-09-17T09:27:52.948579+0000","last_peered":"2024-09-17T09:27:52.948579+0000","last_clean":"2024-09-17T09:27:52.948579+0000","last_became_active":"2024-09-17T09:26:18.760694+0000","last_became_peered":"2024-09-17T09:26:18.760694+0000","last_unstale":"2024-09-17T09:27:52.948579+0000","last_undegraded":"2024-09-17T09:27:52.948579+0000","last_fullsized":"2024-09-17T09:27:52.948579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:30.174719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757378+0000","last_change":"2024-09-17T09:26:20.773420+0000","last_active":"2024-09-17T09:26:46.757378+0000","last_peered":"2024-09-17T09:26:46.757378+0000","last_clean":"2024-09-17T09:26:46.757378+0000","last_became_active":"2024-09-17T09:26:20.773133+0000","last_became_peered":"2024-09-17T09:26:20.773133+0000","last_unstale":"2024-09-17T09:26:46.757378+0000","last_undegraded":"2024-09-17T09:26:46.757378+0000","last_fullsized":"2024-09-17T09:26:46.757378+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:41.824017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.473996+0000","last_change":"2024-09-17T09:26:16.742909+0000","last_active":"2024-09-17T09:26:46.473996+0000","last_peered":"2024-09-17T09:26:46.473996+0000","last_clean":"2024-09-17T09:26:46.473996+0000","last_became_active":"2024-09-17T09:26:16.742638+0000","last_became_peered":"2024-09-17T09:26:16.742638+0000","last_unstale":"2024-09-17T09:26:46.473996+0000","last_undegraded":"2024-09-17T09:26:46.473996+0000","last_fullsized":"2024-09-17T09:26:46.473996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:21:01.469453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474227+0000","last_change":"2024-09-17T09:26:14.728150+0000","last_active":"2024-09-17T09:26:46.474227+0000","last_peered":"2024-09-17T09:26:46.474227+0000","last_clean":"2024-09-17T09:26:46.474227+0000","last_became_active":"2024-09-17T09:26:14.727872+0000","last_became_peered":"2024-09-17T09:26:14.727872+0000","last_unstale":"2024-09-17T09:26:46.474227+0000","last_undegraded":"2024-09-17T09:26:46.474227+0000","last_fullsized":"2024-09-17T09:26:46.474227+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:05.726550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273795+0000","last_change":"2024-09-17T09:26:18.760419+0000","last_active":"2024-09-17T09:26:47.273795+0000","last_peered":"2024-09-17T09:26:47.273795+0000","last_clean":"2024-09-17T09:26:47.273795+0000","last_became_active":"2024-09-17T09:26:18.760107+0000","last_became_peered":"2024-09-17T09:26:18.760107+0000","last_unstale":"2024-09-17T09:26:47.273795+0000","last_undegraded":"2024-09-17T09:26:47.273795+0000","last_fullsized":"2024-09-17T09:26:47.273795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:46.209805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871489+0000","last_change":"2024-09-17T09:26:21.249932+0000","last_active":"2024-09-17T09:26:46.871489+0000","last_peered":"2024-09-17T09:26:46.871489+0000","last_clean":"2024-09-17T09:26:46.871489+0000","last_became_active":"2024-09-17T09:26:21.249590+0000","last_became_peered":"2024-09-17T09:26:21.249590+0000","last_unstale":"2024-09-17T09:26:46.871489+0000","last_undegraded":"2024-09-17T09:26:46.871489+0000","last_fullsized":"2024-09-17T09:26:46.871489+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:29:34.357306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.223218+0000","last_change":"2024-09-17T09:26:14.722875+0000","last_active":"2024-09-17T09:26:47.223218+0000","last_peered":"2024-09-17T09:26:47.223218+0000","last_clean":"2024-09-17T09:26:47.223218+0000","last_became_active":"2024-09-17T09:26:14.722614+0000","last_became_peered":"2024-09-17T09:26:14.722614+0000","last_unstale":"2024-09-17T09:26:47.223218+0000","last_undegraded":"2024-09-17T09:26:47.223218+0000","last_fullsized":"2024-09-17T09:26:47.223218+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:32:36.214730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"67'5","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:47.273675+0000","last_change":"2024-09-17T09:26:16.745725+0000","last_active":"2024-09-17T09:26:47.273675+0000","last_peered":"2024-09-17T09:26:47.273675+0000","last_clean":"2024-09-17T09:26:47.273675+0000","last_became_active":"2024-09-17T09:26:16.745523+0000","last_became_peered":"2024-09-17T09:26:16.745523+0000","last_unstale":"2024-09-17T09:26:47.273675+0000","last_undegraded":"2024-09-17T09:26:47.273675+0000","last_fullsized":"2024-09-17T09:26:47.273675+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:58.930892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.757494+0000","last_change":"2024-09-17T09:26:18.764946+0000","last_active":"2024-09-17T09:26:46.757494+0000","last_peered":"2024-09-17T09:26:46.757494+0000","last_clean":"2024-09-17T09:26:46.757494+0000","last_became_active":"2024-09-17T09:26:18.764653+0000","last_became_peered":"2024-09-17T09:26:18.764653+0000","last_unstale":"2024-09-17T09:26:46.757494+0000","last_undegraded":"2024-09-17T09:26:46.757494+0000","last_fullsized":"2024-09-17T09:26:46.757494+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:04:59.060718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474914+0000","last_change":"2024-09-17T09:26:21.250320+0000","last_active":"2024-09-17T09:26:46.474914+0000","last_peered":"2024-09-17T09:26:46.474914+0000","last_clean":"2024-09-17T09:26:46.474914+0000","last_became_active":"2024-09-17T09:26:21.249995+0000","last_became_peered":"2024-09-17T09:26:21.249995+0000","last_unstale":"2024-09-17T09:26:46.474914+0000","last_undegraded":"2024-09-17T09:26:46.474914+0000","last_fullsized":"2024-09-17T09:26:46.474914+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:19.747616+0000","last_clean_scrub_stamp":"2024-09-17T09:26:19.747616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:22:22.197956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.474644+0000","last_change":"2024-09-17T09:26:14.728071+0000","last_active":"2024-09-17T09:26:46.474644+0000","last_peered":"2024-09-17T09:26:46.474644+0000","last_clean":"2024-09-17T09:26:46.474644+0000","last_became_active":"2024-09-17T09:26:14.727725+0000","last_became_peered":"2024-09-17T09:26:14.727725+0000","last_unstale":"2024-09-17T09:26:46.474644+0000","last_undegraded":"2024-09-17T09:26:46.474644+0000","last_fullsized":"2024-09-17T09:26:46.474644+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:13.703308+0000","last_clean_scrub_stamp":"2024-09-17T09:26:13.703308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:31.258539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"67'9","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871459+0000","last_change":"2024-09-17T09:26:16.812384+0000","last_active":"2024-09-17T09:26:46.871459+0000","last_peered":"2024-09-17T09:26:46.871459+0000","last_clean":"2024-09-17T09:26:46.871459+0000","last_became_active":"2024-09-17T09:26:16.812072+0000","last_became_peered":"2024-09-17T09:26:16.812072+0000","last_unstale":"2024-09-17T09:26:46.871459+0000","last_undegraded":"2024-09-17T09:26:46.871459+0000","last_fullsized":"2024-09-17T09:26:46.871459+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:15.715161+0000","last_clean_scrub_stamp":"2024-09-17T09:26:15.715161+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:46:10.810106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-17T09:26:46.871155+0000","last_change":"2024-09-17T09:26:18.759531+0000","last_active":"2024-09-17T09:26:46.871155+0000","last_peered":"2024-09-17T09:26:46.871155+0000","last_clean":"2024-09-17T09:26:46.871155+0000","last_became_active":"2024-09-17T09:26:18.758378+0000","last_became_peered":"2024-09-17T09:26:18.758378+0000","last_unstale":"2024-09-17T09:26:46.871155+0000","last_undegraded":"2024-09-17T09:26:46.871155+0000","last_fullsized":"2024-09-17T09:26:46.871155+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T09:26:17.731921+0000","last_clean_scrub_stamp":"2024-09-17T09:26:17.731921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:24:26.845256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":92,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":701,"num_read_kb":458,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234010,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32488,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708568,"statfs":{"total":95990841344,"available":95957573632,"internally_reserved":0,"allocated":1409024,"data_stored":922276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41999999999999998}]},{"osd":1,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.38500000000000001}]},{"osd":3,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.44},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.33200000000000002}]},{"osd":4,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.35799999999999998}]},{"osd":5,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.39400000000000002}]},{"osd":6,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":4.2670000000000003,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":0.312},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.40899999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495647,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32452,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708604,"statfs":{"total":95990841344,"available":95957610496,"internally_reserved":0,"allocated":1368064,"data_stored":921553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":0.68700000000000006,"15min":0.57599999999999996},"min":{"1min":0.32800000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":16.181000000000001,"5min":16.181000000000001,"15min":16.181000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.502}]},{"osd":1,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":0.99399999999999999,"15min":0.93600000000000005},"min":{"1min":0.311,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":16.27,"5min":16.27,"15min":16.27},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.794,"5min":3.3159999999999998,"15min":3.3159999999999998},"last":0.48999999999999999}]},{"osd":2,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51500000000000001,"15min":0.504},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.63400000000000001},"min":{"1min":0.39100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":0.46700000000000003}]},{"osd":3,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.59999999999999998,"15min":0.61899999999999999},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66000000000000003,"5min":3.3330000000000002,"15min":3.3330000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.52700000000000002,"15min":0.51500000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40000000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.496,"15min":0.51100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54800000000000004,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Sep 17 09:27:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.43099999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.44400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.63400000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757285,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31888,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709168,"statfs":{"total":95990841344,"available":95958188032,"internally_reserved":0,"allocated":790528,"data_stored":331918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.65100000000000002,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.63},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.502,"15min":0.503},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.48299999999999998,"15min":0.47299999999999998},"min":{"1min":0.38400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.51400000000000001,"15min":0.496},"min":{"1min":0.40300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46000000000000002}]},{"osd":3,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.371,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52700000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.59599999999999997}]},{"osd":4,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.32000000000000001,"5min":0.251,"15min":0.251},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.442},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.40000000000000002,"15min":0.38800000000000001},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Sep 17 09:27:38 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.55900000000000005,"15min":0.57699999999999996},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64300000000000002,"5min":2.496,"15min":2.496},"last":0.502},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45500000000000002,"15min":0.45300000000000001},"min":{"1min":0.35099999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66800000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.372}]},{"osd":7,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":0.40899999999999997}]}]},{"osd":4,"up_from":30,"seq":128849018922,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31916,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709140,"statfs":{"total":95990841344,"available":95958159360,"internally_reserved":0,"allocated":819200,"data_stored":331226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.66300000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.222}]},{"osd":1,"last update":"Tue Sep 17 09:27:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.318,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68700000000000006,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.318},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52100000000000002,"15min":0.52800000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.248}]},{"osd":2,"last update":"Tue Sep 17 09:27:56 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53200000000000003,"15min":0.53900000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79000000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.26300000000000001}]},{"osd":3,"last update":"Tue Sep 17 09:27:56 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81999999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73799999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.36199999999999999}]},{"osd":5,"last update":"Tue Sep 17 09:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.36899999999999999,"15min":0.35699999999999998},"min":{"1min":0.27700000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.42799999999999999,"15min":0.41099999999999998},"min":{"1min":0.316,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.44800000000000001}]},{"osd":6,"last update":"Tue Sep 17 09:27:39 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46100000000000002,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76900000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.41399999999999998}]},{"osd":7,"last update":"Tue Sep 17 09:27:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.371},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40100000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182447,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713280,"statfs":{"total":95990841344,"available":95962398720,"internally_reserved":0,"allocated":774144,"data_stored":330655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.313},{"interface":"front","average":{"1min":0.373,"5min":0.40600000000000003,"15min":0.41499999999999998},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.58899999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.29699999999999999}]},{"osd":1,"last update":"Tue Sep 17 09:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.41999999999999998,"15min":0.435},"min":{"1min":0.27000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.5,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.41199999999999998,"15min":0.42999999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.48099999999999998,"5min":0.625,"15min":0.625},"last":0.23699999999999999}]},{"osd":2,"last update":"Tue Sep 17 09:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.44800000000000001,"15min":0.46500000000000002},"min":{"1min":0.255,"5min":0.252,"15min":0.252},"max":{"1min":0.59199999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.28299999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.55600000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.28100000000000003}]},{"osd":4,"last update":"Tue Sep 17 09:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56100000000000005,"15min":0.57199999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55900000000000005,"15min":0.56499999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.45000000000000001}]},{"osd":5,"last update":"Tue Sep 17 09:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Sep 17 09:27:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78100000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.46200000000000002}]},{"osd":7,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.436}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713264,"statfs":{"total":95990841344,"available":95962382336,"internally_reserved":0,"allocated":790528,"data_stored":331206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34999999999999998,"15min":0.34200000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54400000000000004,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.38100000000000001,"15min":0.36699999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.30599999999999999}]},{"osd":1,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.374,"15min":0.35399999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.54300000000000004,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.36799999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56599999999999995,"5min":0.63,"15min":0.63},"last":0.32500000000000001}]},{"osd":3,"last update":"Tue Sep 17 09:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.438,"15min":0.442},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55400000000000005,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.437,"15min":0.44500000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63400000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue Sep 17 09:27:51 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73699999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.44},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.56000000000000005,"15min":0.57899999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Sep 17 09:27:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.76000000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61499999999999999,"15min":0.626},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":1.012,"15min":1.012},"last":0.52100000000000002}]},{"osd":6,"last update":"Tue Sep 17 09:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.753,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.747,"5min":0.871,"15min":0.871},"last":0.49299999999999999}]},{"osd":7,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.505}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27796,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713260,"statfs":{"total":95990841344,"available":95962378240,"internally_reserved":0,"allocated":794624,"data_stored":331473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 17 09:27:42 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.32900000000000001,"15min":0.311},"min":{"1min":0.20599999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.45100000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.253},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"min":{"1min":0.223,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52000000000000002,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.375}]},{"osd":2,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.35399999999999998,"15min":0.34000000000000002},"min":{"1min":0.23100000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46999999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57299999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.39900000000000002}]},{"osd":3,"last update":"Tue Sep 17 09:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39600000000000002,"15min":0.40200000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41999999999999998,"15min":0.42199999999999999},"min":{"1min":0.29599999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.308}]},{"osd":4,"last update":"Tue Sep 17 09:27:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.39600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.64600000000000002}]},{"osd":5,"last update":"Tue Sep 17 09:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79600000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.75700000000000001}]},{"osd":6,"last update":"Tue Sep 17 09:27:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.39500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Sep 17 09:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.77900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28372,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712684,"statfs":{"total":95990841344,"available":95961788416,"internally_reserved":0,"allocated":1384448,"data_stored":921291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 17 09:27:36 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.33800000000000002,"15min":0.32300000000000001},"min":{"1min":0.23000000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.47699999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42999999999999999,"15min":0.39200000000000002},"min":{"1min":0.315,"5min":0.245,"15min":0.245},"max":{"1min":0.63900000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Sep 17 09:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.375,"15min":0.36499999999999999},"min":{"1min":0.28000000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.308},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.38},"min":{"1min":0.33600000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.505,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.33200000000000002}]},{"osd":3,"last update":"Tue Sep 17 09:27:28 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.441,"15min":0.44900000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.622,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.42099999999999999,"15min":0.42699999999999999},"min":{"1min":0.26200000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.318}]},{"osd":4,"last update":"Tue Sep 17 09:27:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.44}]},{"osd":5,"last update":"Tue Sep 17 09:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.502},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78300000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.47399999999999998}]},{"osd":6,"last update":"Tue Sep 17 09:27:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55300000000000005,"15min":0.56000000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.629,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Sep 17 09:27:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.48499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T09:28:03.086 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-17T09:28:03.086 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T09:28:03.086 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-17T09:28:03.086 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-17T09:28:03.231 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:03 smithi157 ceph-mon[27402]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:28:03.231 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:03 smithi157 ceph-mon[27402]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:03.231 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:03 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:03.231 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:03 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:03.309 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:03 smithi148 ceph-mon[25650]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:28:03.309 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:03 smithi148 ceph-mon[25650]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:03.309 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:03 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:03.309 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:03 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:03.309 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:03 smithi148 ceph-mon[32020]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-17T09:28:03.309 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:03 smithi148 ceph-mon[32020]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:03.309 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:03 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:03.309 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:03 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:04.443 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:28:04.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:04 smithi148 ceph-mon[25650]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:04.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:04 smithi148 ceph-mon[32020]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:04.940 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:04 smithi157 ceph-mon[27402]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:06.483 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:06 smithi148 ceph-mon[25650]: Detected new or changed devices on smithi157 2024-09-17T09:28:06.483 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:06 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:06.483 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:06 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:06.483 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:06 smithi148 ceph-mon[32020]: Detected new or changed devices on smithi157 2024-09-17T09:28:06.483 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:06 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:06.483 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:06 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:06.705 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:06 smithi157 ceph-mon[27402]: Detected new or changed devices on smithi157 2024-09-17T09:28:06.705 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:06 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:06.705 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:06 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:06.802 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-17T09:28:06.803 INFO:teuthology.orchestra.run.smithi148.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T09:28:07.651 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-17T09:28:07.651 INFO:tasks.cephadm:Setup complete, yielding 2024-09-17T09:28:07.651 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-17T09:28:07.661 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi148.front.sepia.ceph.com 2024-09-17T09:28:07.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-09-17T09:28:07.686 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:07 smithi148 ceph-mon[25650]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:07.686 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:07 smithi148 ceph-mon[25650]: from='client.? 172.21.15.148:0/2188253322' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T09:28:07.686 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:07 smithi148 ceph-mon[32020]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:07.686 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:07 smithi148 ceph-mon[32020]: from='client.? 172.21.15.148:0/2188253322' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T09:28:07.730 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:07 smithi157 ceph-mon[27402]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:07.730 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:07 smithi157 ceph-mon[27402]: from='client.? 172.21.15.148:0/2188253322' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-17T09:28:08.776 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:08 smithi157 ceph-mon[27402]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:08.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:08 smithi148 ceph-mon[25650]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:08.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:08 smithi148 ceph-mon[32020]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:09.947 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:28:10.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:28:09 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:28:10.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ::ffff:172.21.15.157 - - [17/Sep/2024:09:28:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:28:11.018 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:11.018 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[25650]: Detected new or changed devices on smithi148 2024-09-17T09:28:11.018 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:11.018 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:11.018 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:28:11.018 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:28:11.019 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[25650]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T09:28:11.019 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:11.019 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[25650]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T09:28:11.019 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[25650]: Reconfiguring daemon alertmanager.a on smithi148 2024-09-17T09:28:11.019 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:11.019 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[32020]: Detected new or changed devices on smithi148 2024-09-17T09:28:11.019 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:11.019 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:11.019 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:28:11.019 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:28:11.019 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[32020]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T09:28:11.020 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:11.020 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[32020]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T09:28:11.020 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:10 smithi148 ceph-mon[32020]: Reconfiguring daemon alertmanager.a on smithi148 2024-09-17T09:28:11.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:10 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:11.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:10 smithi157 ceph-mon[27402]: Detected new or changed devices on smithi148 2024-09-17T09:28:11.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:10 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:11.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:10 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:11.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:10 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:28:11.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:10 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:28:11.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:10 smithi157 ceph-mon[27402]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T09:28:11.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:10 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:11.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:10 smithi157 ceph-mon[27402]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-17T09:28:11.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:10 smithi157 ceph-mon[27402]: Reconfiguring daemon alertmanager.a on smithi148 2024-09-17T09:28:12.162 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:11 smithi148 ceph-mon[25650]: Health check failed: 2 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-17T09:28:12.163 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:11 smithi148 ceph-mon[32020]: Health check failed: 2 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-17T09:28:12.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:11 smithi157 ceph-mon[27402]: Health check failed: 2 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-17T09:28:12.376 INFO:teuthology.orchestra.run.smithi148.stdout:167 167 2024-09-17T09:28:13.029 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-17T09:28:13.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:12 smithi148 ceph-mon[25650]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:13.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:12 smithi148 ceph-mon[32020]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:13.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:12 smithi157 ceph-mon[27402]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:14.475 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:28:14.717 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:14 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:14.717 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:14 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:28:14.717 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:14 smithi148 ceph-mon[25650]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:14.717 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:14 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:14.717 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:14 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:28:14.717 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:14 smithi148 ceph-mon[32020]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:15.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:14 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:15.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:14 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:28:15.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:14 smithi157 ceph-mon[27402]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:17.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:16 smithi157 ceph-mon[27402]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:17.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:16 smithi148 ceph-mon[25650]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:17.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:16 smithi148 ceph-mon[32020]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:17.418 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:17 smithi148 systemd[1]: Stopping Ceph alertmanager.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:28:18.179 INFO:teuthology.orchestra.run.smithi148.stdout:Backend: cephadm 2024-09-17T09:28:18.179 INFO:teuthology.orchestra.run.smithi148.stdout:Available: Yes 2024-09-17T09:28:18.179 INFO:teuthology.orchestra.run.smithi148.stdout:Paused: No 2024-09-17T09:28:18.417 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:18 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[55656]: ts=2024-09-17T09:28:18.145Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-17T09:28:18.836 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:18 smithi148 podman[66879]: 2024-09-17 09:28:18.526905425 +0000 UTC m=+0.501726892 container died 645b209ad43dcef18d66838374808a8d0777e767c4a82c2684b3714fbb6b95a4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:19.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:18 smithi148 ceph-mon[32020]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:19.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:18 smithi148 ceph-mon[32020]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:19.167 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:19 smithi148 podman[66879]: 2024-09-17 09:28:19.043661269 +0000 UTC m=+1.018482767 container cleanup 645b209ad43dcef18d66838374808a8d0777e767c4a82c2684b3714fbb6b95a4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:19.167 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:19 smithi148 bash[66879]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a 2024-09-17T09:28:19.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:18 smithi148 ceph-mon[25650]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:19.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:18 smithi148 ceph-mon[25650]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:19.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:18 smithi157 ceph-mon[27402]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-09-17T09:28:19.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:18 smithi157 ceph-mon[27402]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:19.481 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:19 smithi148 podman[66904]: 2024-09-17 09:28:19.219842966 +0000 UTC m=+0.920057041 container remove 645b209ad43dcef18d66838374808a8d0777e767c4a82c2684b3714fbb6b95a4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:19.481 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:19 smithi148 podman[66904]: 2024-09-17 09:28:19.295221165 +0000 UTC m=+0.995435194 volume remove 4fb51075b48cbb34614d3b1ede3d32041bc42daf8980b23a0db3d41f3d3fd095 2024-09-17T09:28:19.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:28:19 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:28:19.804 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-17T09:28:20.018 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:28:20.139 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:19 smithi148 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-17T09:28:20.139 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:19 smithi148 systemd[1]: Stopped Ceph alertmanager.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:28:20.140 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:19 smithi148 systemd[1]: Starting Ceph alertmanager.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:28:20.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:20 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ::ffff:172.21.15.157 - - [17/Sep/2024:09:28:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:28:21.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:20 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:21.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:20 smithi148 ceph-mon[25650]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T09:28:21.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:20 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:21.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:20 smithi148 ceph-mon[32020]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T09:28:21.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:20 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:21.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:20 smithi157 ceph-mon[27402]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-17T09:28:21.667 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 podman[67069]: 2024-09-17 09:28:21.30460056 +0000 UTC m=+1.246872648 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-17T09:28:21.668 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 podman[67069]: 2024-09-17 09:28:21.406748449 +0000 UTC m=+1.349020589 volume create abb5e38a735b1a914b7313c41207475975c1fe52fe30b81d911e2c5aea58ab7f 2024-09-17T09:28:21.668 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 podman[67069]: 2024-09-17 09:28:21.49033417 +0000 UTC m=+1.432606252 container create c5067a4f15d8779b607189c12be664598ee6c365db6df2206c31c122087b29ed (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:22.067 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 podman[67069]: 2024-09-17 09:28:21.792792171 +0000 UTC m=+1.735064257 container init c5067a4f15d8779b607189c12be664598ee6c365db6df2206c31c122087b29ed (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:22.067 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 podman[67069]: 2024-09-17 09:28:21.795466869 +0000 UTC m=+1.737738948 container start c5067a4f15d8779b607189c12be664598ee6c365db6df2206c31c122087b29ed (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:22.067 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:28:21.807Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-17T09:28:22.067 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:28:21.807Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-17T09:28:22.067 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:28:21.808Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.148 port=9094 2024-09-17T09:28:22.067 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:28:21.809Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-17T09:28:22.068 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:28:21.831Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T09:28:22.068 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:28:21.832Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-17T09:28:22.068 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:28:21.834Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-17T09:28:22.068 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:28:21.834Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-17T09:28:22.068 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 bash[67069]: c5067a4f15d8779b607189c12be664598ee6c365db6df2206c31c122087b29ed 2024-09-17T09:28:22.068 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:21 smithi148 systemd[1]: Started Ceph alertmanager.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:28:22.622 INFO:teuthology.orchestra.run.smithi148.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T09:28:22.622 INFO:teuthology.orchestra.run.smithi148.stdout:agent.smithi148 smithi148 error 21s ago 6m - - 2024-09-17T09:28:22.622 INFO:teuthology.orchestra.run.smithi148.stdout:agent.smithi157 smithi157 error 21s ago 6m - - 2024-09-17T09:28:22.622 INFO:teuthology.orchestra.run.smithi148.stdout:alertmanager.a smithi148 *:9093,9094 starting - - - - 2024-09-17T09:28:22.622 INFO:teuthology.orchestra.run.smithi148.stdout:grafana.a smithi157 *:3000 running (56s) 21s ago 55s 78.5M - 9.4.7 954c08fa6188 3c855b0d97f6 2024-09-17T09:28:22.622 INFO:teuthology.orchestra.run.smithi148.stdout:iscsi.iscsi.a smithi157 *:5000 running (115s) 21s ago 114s 51.1M - 3.8 980a0bfcd684 c1dd8f7936ff 2024-09-17T09:28:22.622 INFO:teuthology.orchestra.run.smithi148.stdout:mgr.x smithi157 *:8443,8765 running (5m) 21s ago 5m 436M - 18.2.4-855-g19e2a2be 980a0bfcd684 2b87dc5c3396 2024-09-17T09:28:22.622 INFO:teuthology.orchestra.run.smithi148.stdout:mgr.y smithi148 *:9283,8765 running (7m) 21s ago 7m 478M - 18.2.4-855-g19e2a2be 980a0bfcd684 b26839806a72 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:mon.a smithi148 running (7m) 21s ago 7m 50.3M 2048M 18.2.4-855-g19e2a2be 980a0bfcd684 b050b0b432b1 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:mon.b smithi157 running (6m) 21s ago 6m 44.5M 2048M 18.2.4-855-g19e2a2be 980a0bfcd684 58bf671c333f 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:mon.c smithi148 running (5m) 21s ago 5m 43.9M 2048M 18.2.4-855-g19e2a2be 980a0bfcd684 6b4438221364 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:node-exporter.a smithi148 *:9100 running (44s) 21s ago 42s 3330k - 1.5.0 0da6a335fe13 e7344f84d46e 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:node-exporter.b smithi157 *:9100 running (39s) 21s ago 39s 3619k - 1.5.0 0da6a335fe13 55850ab76da2 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:osd.0 smithi148 running (5m) 21s ago 5m 48.5M 4096M 18.2.4-855-g19e2a2be 980a0bfcd684 453354340d69 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:osd.1 smithi148 running (4m) 21s ago 4m 46.6M 4096M 18.2.4-855-g19e2a2be 980a0bfcd684 0557b456562f 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:osd.2 smithi148 running (4m) 21s ago 4m 43.5M 4096M 18.2.4-855-g19e2a2be 980a0bfcd684 18327f211771 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:osd.3 smithi148 running (4m) 21s ago 4m 44.9M 4096M 18.2.4-855-g19e2a2be 980a0bfcd684 e784eb75d046 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:osd.4 smithi157 running (3m) 21s ago 3m 53.0M 4096M 18.2.4-855-g19e2a2be 980a0bfcd684 f438f75a9020 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:osd.5 smithi157 running (3m) 21s ago 3m 51.0M 4096M 18.2.4-855-g19e2a2be 980a0bfcd684 20e4681fb2f5 2024-09-17T09:28:22.623 INFO:teuthology.orchestra.run.smithi148.stdout:osd.6 smithi157 running (2m) 21s ago 2m 52.7M 4096M 18.2.4-855-g19e2a2be 980a0bfcd684 91eae7c18e85 2024-09-17T09:28:22.624 INFO:teuthology.orchestra.run.smithi148.stdout:osd.7 smithi157 running (2m) 21s ago 2m 50.0M 4096M 18.2.4-855-g19e2a2be 980a0bfcd684 a97be771a25d 2024-09-17T09:28:22.624 INFO:teuthology.orchestra.run.smithi148.stdout:prometheus.a smithi157 *:9095 running (90s) 21s ago 90s 28.9M - 2.43.0 a07b618ecd1d 58bc01a404c8 2024-09-17T09:28:22.624 INFO:teuthology.orchestra.run.smithi148.stdout:rgw.foo.a smithi148 *:80 running (2m) 21s ago 2m 74.8M - 18.2.4-855-g19e2a2be 980a0bfcd684 a7ed1a8a3f96 2024-09-17T09:28:23.023 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:22 smithi157 ceph-mon[27402]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:23.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:22 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:23.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:22 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:23.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:22 smithi157 ceph-mon[27402]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T09:28:23.024 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:22 smithi157 ceph-mon[27402]: Reconfiguring daemon prometheus.a on smithi157 2024-09-17T09:28:23.318 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:22 smithi148 ceph-mon[25650]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:23.318 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:22 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:23.318 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:22 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:23.319 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:22 smithi148 ceph-mon[25650]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T09:28:23.319 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:22 smithi148 ceph-mon[25650]: Reconfiguring daemon prometheus.a on smithi157 2024-09-17T09:28:23.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:22 smithi148 ceph-mon[32020]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:23.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:22 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:23.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:22 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:23.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:22 smithi148 ceph-mon[32020]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-17T09:28:23.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:22 smithi148 ceph-mon[32020]: Reconfiguring daemon prometheus.a on smithi157 2024-09-17T09:28:23.339 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-17T09:28:23.917 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:23 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:28:23.809Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000023862s 2024-09-17T09:28:24.199 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:23 smithi148 ceph-mon[25650]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:24.200 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:23 smithi148 ceph-mon[32020]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:24.231 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:23 smithi157 ceph-mon[27402]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:24.299 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:28:25.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:24 smithi157 ceph-mon[27402]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:25.274 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 systemd[1]: Stopping Ceph prometheus.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:28:25.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:24 smithi148 ceph-mon[25650]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:25.483 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:24 smithi148 ceph-mon[32020]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-17T09:28:25.657 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.363Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-17T09:28:25.657 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.363Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-17T09:28:25.657 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.363Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-17T09:28:25.657 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.363Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-17T09:28:25.657 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.363Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-17T09:28:25.657 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.363Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-17T09:28:25.657 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.363Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-17T09:28:25.657 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.363Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-17T09:28:25.657 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.364Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-17T09:28:25.657 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.364Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-17T09:28:25.658 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.364Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-17T09:28:25.658 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[49461]: ts=2024-09-17T09:28:25.364Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-17T09:28:25.658 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 podman[60787]: 2024-09-17 09:28:25.374037276 +0000 UTC m=+0.146622519 container died 58bc01a404c82bb1d9fde8148fa36da721c4721d7f35b493584d1e089decdc6c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:25.924 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 podman[60787]: 2024-09-17 09:28:25.762868333 +0000 UTC m=+0.535453554 container cleanup 58bc01a404c82bb1d9fde8148fa36da721c4721d7f35b493584d1e089decdc6c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:25.924 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:25 smithi157 bash[60787]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a 2024-09-17T09:28:26.168 INFO:teuthology.orchestra.run.smithi148.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-17T09:28:26.169 INFO:teuthology.orchestra.run.smithi148.stdout:agent 0/2 25s ago 87s * 2024-09-17T09:28:26.169 INFO:teuthology.orchestra.run.smithi148.stdout:alertmanager ?:9093,9094 0/1 - 109s smithi148=a;count:1 2024-09-17T09:28:26.169 INFO:teuthology.orchestra.run.smithi148.stdout:grafana ?:3000 1/1 25s ago 106s smithi157=a;count:1 2024-09-17T09:28:26.169 INFO:teuthology.orchestra.run.smithi148.stdout:iscsi.datapool ?:5000 1/1 25s ago 2m smithi157=iscsi.a;count:1 2024-09-17T09:28:26.169 INFO:teuthology.orchestra.run.smithi148.stdout:mgr 2/2 25s ago 5m smithi148=y;smithi157=x;count:2 2024-09-17T09:28:26.169 INFO:teuthology.orchestra.run.smithi148.stdout:mon 3/3 25s ago 6m smithi148:172.21.15.148=a;smithi148:[v2:172.21.15.148:3301,v1:172.21.15.148:6790]=c;smithi157:172.21.15.157=b;count:3 2024-09-17T09:28:26.169 INFO:teuthology.orchestra.run.smithi148.stdout:node-exporter ?:9100 2/2 25s ago 113s smithi148=a;smithi157=b;count:2 2024-09-17T09:28:26.169 INFO:teuthology.orchestra.run.smithi148.stdout:osd 8 25s ago - 2024-09-17T09:28:26.169 INFO:teuthology.orchestra.run.smithi148.stdout:prometheus ?:9095 1/1 25s ago 118s smithi157=a;count:1 2024-09-17T09:28:26.169 INFO:teuthology.orchestra.run.smithi148.stdout:rgw.foo.a ?:80 1/1 25s ago 2m smithi148=foo.a;count:1 2024-09-17T09:28:26.211 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:26 smithi157 podman[60801]: 2024-09-17 09:28:26.006591279 +0000 UTC m=+0.630600479 container remove 58bc01a404c82bb1d9fde8148fa36da721c4721d7f35b493584d1e089decdc6c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:26.524 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:26 smithi157 podman[60823]: 2024-09-17 09:28:26.401077125 +0000 UTC m=+0.617713571 container died b93ae8224f2dbe2cca14c0dcee6b4a96c89079fdb76411cb790a265940b1d20d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=jovial_jennings, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-09-17T09:28:26.926 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-17T09:28:27.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:26 smithi148 ceph-mon[25650]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:27.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:26 smithi148 ceph-mon[25650]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:27.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:26 smithi148 ceph-mon[32020]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:27.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:26 smithi148 ceph-mon[32020]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:27.247 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:26 smithi157 ceph-mon[27402]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:27.247 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:26 smithi157 ceph-mon[27402]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:27.523 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-17T09:28:27.524 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 systemd[1]: Stopped Ceph prometheus.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:28:27.524 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.243s CPU time. 2024-09-17T09:28:27.524 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 systemd[1]: Starting Ceph prometheus.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:28:27.524 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 podman[60962]: 2024-09-17 09:28:27.376274969 +0000 UTC m=+0.016864341 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-17T09:28:27.524 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 podman[60962]: 2024-09-17 09:28:27.489683342 +0000 UTC m=+0.130272728 container create c87d17afe318f05fa60790b07dee95f52048b47489905e007dc53e848edfd0fc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:28.018 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 podman[60962]: 2024-09-17 09:28:27.733220982 +0000 UTC m=+0.373810340 container init c87d17afe318f05fa60790b07dee95f52048b47489905e007dc53e848edfd0fc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:28.018 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 podman[60962]: 2024-09-17 09:28:27.735900043 +0000 UTC m=+0.376489384 container start c87d17afe318f05fa60790b07dee95f52048b47489905e007dc53e848edfd0fc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.760Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.760Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.760Z caller=main.go:570 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi157 (none))" 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.760Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.760Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.762Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.762Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.763Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.763Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.765Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.765Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.757µs 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.765Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-17T09:28:28.019 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.765Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-17T09:28:28.020 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.766Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-17T09:28:28.020 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.766Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.503µs wal_replay_duration=1.116887ms wbl_replay_duration=151ns total_replay_duration=1.151798ms 2024-09-17T09:28:28.020 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.767Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-17T09:28:28.020 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.767Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-17T09:28:28.020 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.767Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-17T09:28:28.020 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.786Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=19.193976ms db_storage=753ns remote_storage=1.329µs web_handler=351ns query_engine=519ns scrape=498.474µs scrape_sd=102.741µs notify=10.657µs notify_sd=12.116µs rules=18.219961ms tracing=5.639µs 2024-09-17T09:28:28.020 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.786Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-17T09:28:28.020 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:28:27.786Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-17T09:28:28.020 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 bash[60962]: c87d17afe318f05fa60790b07dee95f52048b47489905e007dc53e848edfd0fc 2024-09-17T09:28:28.020 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:28:27 smithi157 systemd[1]: Started Ceph prometheus.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:28:28.098 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:28:28.167 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:27 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:27] ENGINE Bus STOPPING 2024-09-17T09:28:28.478 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:28] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T09:28:28.479 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:28] ENGINE Bus STOPPED 2024-09-17T09:28:28.479 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:28] ENGINE Bus STARTING 2024-09-17T09:28:28.479 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:28] ENGINE Serving on http://:::9283 2024-09-17T09:28:28.479 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:28] ENGINE Bus STARTED 2024-09-17T09:28:28.791 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:28] ENGINE Bus STOPPING 2024-09-17T09:28:29.086 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:28] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T09:28:29.086 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:28] ENGINE Bus STOPPED 2024-09-17T09:28:29.086 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:28] ENGINE Bus STARTING 2024-09-17T09:28:29.087 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.087 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.087 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T09:28:29.087 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T09:28:29.087 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi148.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T09:28:29.087 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi148.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.088 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:28:29.089 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:28:29.089 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:28:29.089 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:29.089 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.089 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[32020]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:28:29.089 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.089 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.090 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T09:28:29.090 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T09:28:29.090 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi148.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T09:28:29.090 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi148.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T09:28:29.090 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.091 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T09:28:29.091 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T09:28:29.091 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T09:28:29.091 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T09:28:29.091 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.091 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T09:28:29.091 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T09:28:29.092 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T09:28:29.092 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T09:28:29.092 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T09:28:29.092 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.092 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:28:29.092 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:28:29.093 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:28:29.093 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:29.093 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.093 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:28 smithi148 ceph-mon[25650]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:28:29.241 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.241 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.241 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T09:28:29.241 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-17T09:28:29.241 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi148.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T09:28:29.242 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi148.front.sepia.ceph.com:9093"}]: dispatch 2024-09-17T09:28:29.242 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.242 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T09:28:29.242 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-17T09:28:29.242 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T09:28:29.242 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-09-17T09:28:29.242 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.242 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-17T09:28:29.242 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T09:28:29.242 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-17T09:28:29.242 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T09:28:29.243 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-09-17T09:28:29.243 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.243 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T09:28:29.243 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T09:28:29.243 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T09:28:29.243 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:29.243 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' 2024-09-17T09:28:29.243 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:28 smithi157 ceph-mon[27402]: from='mgr.14610 172.21.15.148:0/334113045' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T09:28:29.417 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:29] ENGINE Serving on http://:::9283 2024-09-17T09:28:29.417 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:29] ENGINE Bus STARTED 2024-09-17T09:28:29.417 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:29] ENGINE Bus STOPPING 2024-09-17T09:28:29.840 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:29] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T09:28:29.840 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:29] ENGINE Bus STOPPED 2024-09-17T09:28:29.840 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:29] ENGINE Bus STARTING 2024-09-17T09:28:29.840 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:29] ENGINE Serving on http://:::9283 2024-09-17T09:28:29.841 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:29] ENGINE Bus STARTED 2024-09-17T09:28:29.860 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:28:29 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:28:30.167 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-mon[25650]: Health check cleared: CEPHADM_AGENT_DOWN (was: 2 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-17T09:28:30.168 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-mon[25650]: Cluster is now healthy 2024-09-17T09:28:30.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-mon[32020]: Health check cleared: CEPHADM_AGENT_DOWN (was: 2 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-17T09:28:30.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:29 smithi148 ceph-mon[32020]: Cluster is now healthy 2024-09-17T09:28:30.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:29 smithi157 ceph-mon[27402]: Health check cleared: CEPHADM_AGENT_DOWN (was: 2 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-17T09:28:30.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:29 smithi157 ceph-mon[27402]: Cluster is now healthy 2024-09-17T09:28:30.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:30 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:30.917 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:30 smithi148 ceph-mon[25650]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:30.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:30 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:30.918 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:30 smithi148 ceph-mon[32020]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:31.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:30 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:31.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:30 smithi157 ceph-mon[27402]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:31.498 INFO:teuthology.orchestra.run.smithi148.stdout:HOST ADDR LABELS STATUS 2024-09-17T09:28:31.499 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148 172.21.15.148 2024-09-17T09:28:31.499 INFO:teuthology.orchestra.run.smithi148.stdout:smithi157 172.21.15.157 2024-09-17T09:28:31.499 INFO:teuthology.orchestra.run.smithi148.stdout:2 hosts in cluster 2024-09-17T09:28:32.076 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:31 smithi148 ceph-mon[32020]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:32.077 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:31 smithi148 ceph-mon[25650]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:32.077 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:28:31 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:28:31.811Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001751469s 2024-09-17T09:28:32.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:31 smithi157 ceph-mon[27402]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:32.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-17T09:28:33.018 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:32 smithi148 ceph-mon[25650]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:33.019 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:32 smithi148 ceph-mon[32020]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:33.173 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:28:33.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:32 smithi157 ceph-mon[27402]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:35.121 INFO:teuthology.orchestra.run.smithi148.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-17T09:28:35.121 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005G400BGN 372G No 25s ago Has a FileSystem, LVM detected 2024-09-17T09:28:35.121 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148 /dev/nvme1n1 ssd Linux_da21824e67705baa6884 89.4G No 25s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T09:28:35.121 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148 /dev/nvme2n1 ssd Linux_4379e416df77cf91daa0 89.4G No 25s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T09:28:35.121 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148 /dev/nvme3n1 ssd Linux_12b1430a50ac941ee93e 89.4G No 25s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T09:28:35.122 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148 /dev/nvme4n1 ssd Linux_0a446a8efc652f5d297c 89.4G No 25s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T09:28:35.122 INFO:teuthology.orchestra.run.smithi148.stdout:smithi157 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002V480DGN 447G No 29s ago Has a FileSystem, LVM detected 2024-09-17T09:28:35.122 INFO:teuthology.orchestra.run.smithi148.stdout:smithi157 /dev/nvme1n1 ssd Linux_6bc07644d91991691c5d 89.4G No 29s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T09:28:35.122 INFO:teuthology.orchestra.run.smithi148.stdout:smithi157 /dev/nvme2n1 ssd Linux_67094b2ea30257741a9e 89.4G No 29s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T09:28:35.122 INFO:teuthology.orchestra.run.smithi148.stdout:smithi157 /dev/nvme3n1 ssd Linux_2ed25dbc61f73a85ec5b 89.4G No 29s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T09:28:35.122 INFO:teuthology.orchestra.run.smithi148.stdout:smithi157 /dev/nvme4n1 ssd Linux_fd032744ff9608c07997 89.4G No 29s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-17T09:28:35.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:35 smithi157 ceph-mon[27402]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:35.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:35 smithi148 ceph-mon[25650]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:35.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:35 smithi148 ceph-mon[32020]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:35.841 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls --format yaml' 2024-09-17T09:28:36.043 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:28:36.163 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:36 smithi148 ceph-mon[25650]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:36.164 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:36 smithi148 ceph-mon[32020]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:36.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:36 smithi157 ceph-mon[27402]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:37.075 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:37 smithi157 ceph-mon[27402]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:37.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:37 smithi148 ceph-mon[25650]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:37.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:37 smithi148 ceph-mon[32020]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:37.932 INFO:teuthology.orchestra.run.smithi148.stdout:service_type: agent 2024-09-17T09:28:37.932 INFO:teuthology.orchestra.run.smithi148.stdout:service_name: agent 2024-09-17T09:28:37.932 INFO:teuthology.orchestra.run.smithi148.stdout:placement: 2024-09-17T09:28:37.932 INFO:teuthology.orchestra.run.smithi148.stdout: host_pattern: '*' 2024-09-17T09:28:37.932 INFO:teuthology.orchestra.run.smithi148.stdout:status: 2024-09-17T09:28:37.932 INFO:teuthology.orchestra.run.smithi148.stdout: created: '2024-09-17T09:26:58.548301Z' 2024-09-17T09:28:37.932 INFO:teuthology.orchestra.run.smithi148.stdout: last_refresh: '2024-09-17T09:28:01.089406Z' 2024-09-17T09:28:37.932 INFO:teuthology.orchestra.run.smithi148.stdout: running: 0 2024-09-17T09:28:37.932 INFO:teuthology.orchestra.run.smithi148.stdout: size: 2 2024-09-17T09:28:37.932 INFO:teuthology.orchestra.run.smithi148.stdout:events: 2024-09-17T09:28:37.932 INFO:teuthology.orchestra.run.smithi148.stdout:- 2024-09-17T09:26:59.944217Z service:agent [INFO] "service was created" 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout:--- 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout:service_type: alertmanager 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout:service_name: alertmanager 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout:placement: 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout: count: 1 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout: hosts: 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi148=a 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout:status: 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout: created: '2024-09-17T09:26:36.211484Z' 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout: ports: 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout: - 9093 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout: - 9094 2024-09-17T09:28:37.933 INFO:teuthology.orchestra.run.smithi148.stdout: running: 0 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout: size: 1 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout:events: 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout:- 2024-09-17T09:27:15.509364Z service:alertmanager [INFO] "service was created" 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout:--- 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout:service_type: grafana 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout:service_name: grafana 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout:placement: 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout: count: 1 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout: hosts: 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi157=a 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout:spec: 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout: anonymous_access: true 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout: protocol: https 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout:status: 2024-09-17T09:28:37.934 INFO:teuthology.orchestra.run.smithi148.stdout: created: '2024-09-17T09:26:39.425148Z' 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout: last_refresh: '2024-09-17T09:28:01.159391Z' 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout: ports: 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout: - 3000 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout: running: 1 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout: size: 1 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout:events: 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout:- 2024-09-17T09:27:26.883960Z service:grafana [INFO] "service was created" 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout:--- 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout:service_type: iscsi 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout:service_id: datapool 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout:service_name: iscsi.datapool 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout:placement: 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout: count: 1 2024-09-17T09:28:37.935 INFO:teuthology.orchestra.run.smithi148.stdout: hosts: 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi157=iscsi.a 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout:spec: 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: api_password: admin 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: api_port: 5000 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: api_user: admin 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: pool: datapool 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: trusted_ip_list: 172.21.15.157 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout:status: 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: created: '2024-09-17T09:26:23.444461Z' 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: last_refresh: '2024-09-17T09:28:01.159272Z' 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: ports: 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: - 5000 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: running: 1 2024-09-17T09:28:37.936 INFO:teuthology.orchestra.run.smithi148.stdout: size: 1 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout:--- 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout:service_type: mgr 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout:service_name: mgr 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout:placement: 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout: count: 2 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout: hosts: 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi148=y 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi157=x 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout:status: 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout: created: '2024-09-17T09:22:38.921927Z' 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout: last_refresh: '2024-09-17T09:28:01.089356Z' 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout: running: 2 2024-09-17T09:28:37.937 INFO:teuthology.orchestra.run.smithi148.stdout: size: 2 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout:--- 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout:service_type: mon 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout:service_name: mon 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout:placement: 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout: count: 3 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout: hosts: 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi148:172.21.15.148=a 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi148:[v2:172.21.15.148:3301,v1:172.21.15.148:6790]=c 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi157:172.21.15.157=b 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout:status: 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout: created: '2024-09-17T09:22:12.818066Z' 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout: last_refresh: '2024-09-17T09:28:01.089244Z' 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout: running: 3 2024-09-17T09:28:37.938 INFO:teuthology.orchestra.run.smithi148.stdout: size: 3 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout:--- 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout:service_type: node-exporter 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout:service_name: node-exporter 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout:placement: 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout: count: 2 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout: hosts: 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi148=a 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi157=b 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout:status: 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout: created: '2024-09-17T09:26:32.639823Z' 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout: last_refresh: '2024-09-17T09:28:01.089977Z' 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout: ports: 2024-09-17T09:28:37.939 INFO:teuthology.orchestra.run.smithi148.stdout: - 9100 2024-09-17T09:28:37.940 INFO:teuthology.orchestra.run.smithi148.stdout: running: 2 2024-09-17T09:28:37.940 INFO:teuthology.orchestra.run.smithi148.stdout: size: 2 2024-09-17T09:28:37.940 INFO:teuthology.orchestra.run.smithi148.stdout:events: 2024-09-17T09:28:37.941 INFO:teuthology.orchestra.run.smithi148.stdout:- 2024-09-17T09:27:43.442197Z service:node-exporter [INFO] "service was created" 2024-09-17T09:28:37.941 INFO:teuthology.orchestra.run.smithi148.stdout:--- 2024-09-17T09:28:37.941 INFO:teuthology.orchestra.run.smithi148.stdout:service_type: osd 2024-09-17T09:28:37.941 INFO:teuthology.orchestra.run.smithi148.stdout:service_name: osd 2024-09-17T09:28:37.941 INFO:teuthology.orchestra.run.smithi148.stdout:unmanaged: true 2024-09-17T09:28:37.941 INFO:teuthology.orchestra.run.smithi148.stdout:spec: 2024-09-17T09:28:37.941 INFO:teuthology.orchestra.run.smithi148.stdout: filter_logic: AND 2024-09-17T09:28:37.941 INFO:teuthology.orchestra.run.smithi148.stdout: objectstore: bluestore 2024-09-17T09:28:37.941 INFO:teuthology.orchestra.run.smithi148.stdout:status: 2024-09-17T09:28:37.941 INFO:teuthology.orchestra.run.smithi148.stdout: container_image_id: 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 2024-09-17T09:28:37.941 INFO:teuthology.orchestra.run.smithi148.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout: last_refresh: '2024-09-17T09:28:01.089563Z' 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout: running: 8 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout: size: 8 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout:--- 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout:service_type: prometheus 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout:service_name: prometheus 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout:placement: 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout: count: 1 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout: hosts: 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi157=a 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout:status: 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout: created: '2024-09-17T09:26:28.012172Z' 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout: ports: 2024-09-17T09:28:37.942 INFO:teuthology.orchestra.run.smithi148.stdout: - 9095 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout: running: 0 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout: size: 1 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout:events: 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout:- 2024-09-17T09:27:43.452163Z service:prometheus [INFO] "service was created" 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout:--- 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout:service_type: rgw 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout:service_id: foo.a 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout:service_name: rgw.foo.a 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout:placement: 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout: count: 1 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout: hosts: 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout: - smithi148=foo.a 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout:status: 2024-09-17T09:28:37.943 INFO:teuthology.orchestra.run.smithi148.stdout: created: '2024-09-17T09:26:13.183762Z' 2024-09-17T09:28:37.944 INFO:teuthology.orchestra.run.smithi148.stdout: last_refresh: '2024-09-17T09:28:01.089829Z' 2024-09-17T09:28:37.944 INFO:teuthology.orchestra.run.smithi148.stdout: ports: 2024-09-17T09:28:37.944 INFO:teuthology.orchestra.run.smithi148.stdout: - 80 2024-09-17T09:28:37.944 INFO:teuthology.orchestra.run.smithi148.stdout: running: 1 2024-09-17T09:28:37.944 INFO:teuthology.orchestra.run.smithi148.stdout: size: 1 2024-09-17T09:28:38.826 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-09-17T09:28:39.035 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/mon.a/config 2024-09-17T09:28:39.157 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:39 smithi148 ceph-mon[25650]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-17T09:28:39.157 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:39 smithi148 ceph-mon[25650]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:39.157 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:39 smithi148 ceph-mon[32020]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-17T09:28:39.157 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:39 smithi148 ceph-mon[32020]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:39.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:39 smithi157 ceph-mon[27402]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-17T09:28:39.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:39 smithi157 ceph-mon[27402]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 0 op/s 2024-09-17T09:28:40.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:28:39 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug there is no tcmu-runner data available 2024-09-17T09:28:40.596 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:40 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: ::ffff:172.21.15.157 - - [17/Sep/2024:09:28:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-17T09:28:40.984 INFO:teuthology.orchestra.run.smithi148.stdout:osd 8 39s ago - 2024-09-17T09:28:41.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:41 smithi157 ceph-mon[27402]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:41.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:41 smithi157 ceph-mon[27402]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:41.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:41 smithi148 ceph-mon[25650]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:41.417 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:41 smithi148 ceph-mon[25650]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:41.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:41 smithi148 ceph-mon[32020]: from='client.24464 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-17T09:28:41.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:41 smithi148 ceph-mon[32020]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-17T09:28:41.621 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-17T09:28:41.631 INFO:tasks.cephadm:Teardown begin 2024-09-17T09:28:41.632 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:28:41.659 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:28:41.696 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-17T09:28:41.696 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T09:28:41.716 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T09:28:41.754 INFO:tasks.cephadm:Stopping all daemons... 2024-09-17T09:28:41.754 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-17T09:28:41.754 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.a 2024-09-17T09:28:42.062 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:42 smithi148 ceph-mon[32020]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:42.063 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:41 smithi148 systemd[1]: Stopping Ceph mon.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:28:42.063 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:42 smithi148 ceph-mon[25650]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:42.412 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:42] ENGINE Bus STOPPING 2024-09-17T09:28:42.412 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a[25646]: 2024-09-17T09:28:42.061+0000 7f3d56092640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:28:42.413 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a[25646]: 2024-09-17T09:28:42.061+0000 7f3d56092640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-17T09:28:42.413 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:42 smithi148 podman[68754]: 2024-09-17 09:28:42.173777576 +0000 UTC m=+0.232748898 container died b050b0b432b1c6651e3ae4517ed4c8331397fed3a0a4dbb5cc09bb8e769f6e73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-17T09:28:42.523 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:42 smithi157 ceph-mon[27402]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T09:28:42.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:42 smithi148 podman[68754]: 2024-09-17 09:28:42.56114412 +0000 UTC m=+0.620115436 container cleanup b050b0b432b1c6651e3ae4517ed4c8331397fed3a0a4dbb5cc09bb8e769f6e73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-17T09:28:42.667 INFO:journalctl@ceph.mon.a.smithi148.stdout:Sep 17 09:28:42 smithi148 bash[68754]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-a 2024-09-17T09:28:42.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T09:28:42.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:42] ENGINE Bus STOPPED 2024-09-17T09:28:42.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:42] ENGINE Bus STARTING 2024-09-17T09:28:42.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:42] ENGINE Serving on http://:::9283 2024-09-17T09:28:42.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:42 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:42] ENGINE Bus STARTED 2024-09-17T09:28:43.024 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.a.service' 2024-09-17T09:28:43.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:28:43.095 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-17T09:28:43.095 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-17T09:28:43.095 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.c 2024-09-17T09:28:43.392 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:43 smithi148 systemd[1]: Stopping Ceph mon.c for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:28:43.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-c[32014]: 2024-09-17T09:28:43.390+0000 7fa9054f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:28:43.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-c[32014]: 2024-09-17T09:28:43.390+0000 7fa9054f9640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-17T09:28:43.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:43 smithi148 podman[69007]: 2024-09-17 09:28:43.46256978 +0000 UTC m=+0.173346287 container died 6b443822136478d7cde4e7d73395e695269d966af27e12e5b964741a06785456 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.license=GPLv2) 2024-09-17T09:28:43.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:43] ENGINE Bus STOPPING 2024-09-17T09:28:43.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:43] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-17T09:28:43.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:43] ENGINE Bus STOPPED 2024-09-17T09:28:43.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:43] ENGINE Bus STARTING 2024-09-17T09:28:43.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:43] ENGINE Serving on http://:::9283 2024-09-17T09:28:43.668 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:43 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: [17/Sep/2024:09:28:43] ENGINE Bus STARTED 2024-09-17T09:28:44.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:43 smithi148 podman[69007]: 2024-09-17 09:28:43.925360072 +0000 UTC m=+0.636136622 container cleanup 6b443822136478d7cde4e7d73395e695269d966af27e12e5b964741a06785456 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-09-17T09:28:44.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:43 smithi148 bash[69007]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-c 2024-09-17T09:28:44.590 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 17 09:28:44 smithi148 podman[69033]: 2024-09-17 09:28:44.303093304 +0000 UTC m=+0.844432072 container remove 6b443822136478d7cde4e7d73395e695269d966af27e12e5b964741a06785456 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-17T09:28:45.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.c.service' 2024-09-17T09:28:45.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:28:45.217 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-17T09:28:45.217 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-17T09:28:45.217 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.b 2024-09-17T09:28:45.503 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:45 smithi157 systemd[1]: Stopping Ceph mon.b for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:28:45.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-b[27393]: 2024-09-17T09:28:45.501+0000 7f081c48a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:28:45.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-b[27393]: 2024-09-17T09:28:45.501+0000 7f081c48a640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-17T09:28:45.773 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:45 smithi157 podman[61788]: 2024-09-17 09:28:45.665436006 +0000 UTC m=+0.253654560 container died 58bf671c333fea30bd71c4f379b44fcffcf8d7f8b331ac1fa6311341f8186f5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:28:46.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:46 smithi157 podman[61788]: 2024-09-17 09:28:46.143635012 +0000 UTC m=+0.731853559 container cleanup 58bf671c333fea30bd71c4f379b44fcffcf8d7f8b331ac1fa6311341f8186f5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, io.buildah.version=1.37.2) 2024-09-17T09:28:46.274 INFO:journalctl@ceph.mon.b.smithi157.stdout:Sep 17 09:28:46 smithi157 bash[61788]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mon-b 2024-09-17T09:28:46.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mon.b.service' 2024-09-17T09:28:46.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:28:46.636 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-17T09:28:46.636 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-17T09:28:46.636 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.y 2024-09-17T09:28:46.917 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:46 smithi148 systemd[1]: Stopping Ceph mgr.y for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:28:48.391 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:48 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:28:48.257+0000 7f156efc6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:28:48.391 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:48 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y[25876]: 2024-09-17T09:28:48.257+0000 7f156efc6640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-17T09:28:48.667 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:48 smithi148 podman[69528]: 2024-09-17 09:28:48.468589179 +0000 UTC m=+1.134166570 container died b26839806a720ed89c916e6345086d78727988d5b8a35253b6a9363275b363aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-17T09:28:49.417 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:49 smithi148 podman[69528]: 2024-09-17 09:28:49.079730826 +0000 UTC m=+1.745308214 container cleanup b26839806a720ed89c916e6345086d78727988d5b8a35253b6a9363275b363aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b) 2024-09-17T09:28:49.417 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:49 smithi148 bash[69528]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y 2024-09-17T09:28:50.763 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-17T09:28:50.786 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:50 smithi148 podman[69584]: 2024-09-17 09:28:50.513990882 +0000 UTC m=+2.220021298 container remove b26839806a720ed89c916e6345086d78727988d5b8a35253b6a9363275b363aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:19e2a2beb6f41a3ab62650e38614eb29b59dcbb9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T09:28:50.786 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:50 smithi148 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.y.service: Deactivated successfully. 2024-09-17T09:28:50.788 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:50 smithi148 systemd[1]: Stopped Ceph mgr.y for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:28:50.788 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 17 09:28:50 smithi148 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.y.service: Consumed 47.149s CPU time. 2024-09-17T09:28:50.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:28:50.803 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-17T09:28:50.804 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-17T09:28:50.804 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.x 2024-09-17T09:28:51.161 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:28:50 smithi157 systemd[1]: Stopping Ceph mgr.x for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:28:51.463 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:28:51 smithi157 podman[62060]: 2024-09-17 09:28:51.159629489 +0000 UTC m=+0.152289073 container died 2b87dc5c3396b9e46d4aa55ec1326fe1323c668158dc5ea1d8d8a5391ecd5d53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b) 2024-09-17T09:28:51.763 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:28:51 smithi157 podman[62060]: 2024-09-17 09:28:51.545249985 +0000 UTC m=+0.537909614 container cleanup 2b87dc5c3396b9e46d4aa55ec1326fe1323c668158dc5ea1d8d8a5391ecd5d53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-17T09:28:51.763 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Sep 17 09:28:51 smithi157 bash[62060]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-mgr-x 2024-09-17T09:28:51.902 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-17T09:28:51.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:28:51.939 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-17T09:28:51.940 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-17T09:28:51.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.0 2024-09-17T09:28:52.226 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:51 smithi148 systemd[1]: Stopping Ceph osd.0 for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:28:52.667 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:52 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0[36240]: 2024-09-17T09:28:52.224+0000 7f0e7ccbc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:28:52.667 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:52 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0[36240]: 2024-09-17T09:28:52.224+0000 7f0e7ccbc640 -1 osd.0 71 *** Got signal Terminated *** 2024-09-17T09:28:52.667 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:52 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0[36240]: 2024-09-17T09:28:52.224+0000 7f0e7ccbc640 -1 osd.0 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T09:28:58.010 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:57 smithi148 podman[69803]: 2024-09-17 09:28:57.723512808 +0000 UTC m=+5.592591649 container died 453354340d6960e67184bd534ea0e33bb52ddf0d6083981ddc4b18d804734a12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-17T09:28:58.417 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:58 smithi148 podman[69803]: 2024-09-17 09:28:58.183265612 +0000 UTC m=+6.052344444 container cleanup 453354340d6960e67184bd534ea0e33bb52ddf0d6083981ddc4b18d804734a12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-17T09:28:58.417 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:58 smithi148 bash[69803]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0 2024-09-17T09:28:58.818 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:58 smithi148 podman[69820]: 2024-09-17 09:28:58.484607663 +0000 UTC m=+0.752392294 container remove 453354340d6960e67184bd534ea0e33bb52ddf0d6083981ddc4b18d804734a12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-17T09:28:59.113 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:58 smithi148 podman[69912]: 2024-09-17 09:28:58.7205927 +0000 UTC m=+0.015877979 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:28:59.114 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:58 smithi148 podman[69912]: 2024-09-17 09:28:58.849118591 +0000 UTC m=+0.144403904 container create d948efad12fd88fefa5858a583846d29f13bee1fa9532a790aac2bd0dd330c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-17T09:28:59.417 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:59 smithi148 podman[69912]: 2024-09-17 09:28:59.113317226 +0000 UTC m=+0.408602500 container init d948efad12fd88fefa5858a583846d29f13bee1fa9532a790aac2bd0dd330c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-17T09:28:59.417 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:59 smithi148 podman[69912]: 2024-09-17 09:28:59.116658541 +0000 UTC m=+0.411943814 container start d948efad12fd88fefa5858a583846d29f13bee1fa9532a790aac2bd0dd330c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-17T09:28:59.418 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:59 smithi148 podman[69912]: 2024-09-17 09:28:59.213403173 +0000 UTC m=+0.508688443 container attach d948efad12fd88fefa5858a583846d29f13bee1fa9532a790aac2bd0dd330c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T09:28:59.418 INFO:journalctl@ceph.osd.0.smithi148.stdout:Sep 17 09:28:59 smithi148 podman[69912]: 2024-09-17 09:28:59.307019159 +0000 UTC m=+0.602304438 container died d948efad12fd88fefa5858a583846d29f13bee1fa9532a790aac2bd0dd330c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-17T09:28:59.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.0.service' 2024-09-17T09:28:59.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:28:59.939 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-17T09:28:59.939 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-17T09:28:59.939 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.1 2024-09-17T09:29:00.167 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:00 smithi148 systemd[1]: Stopping Ceph osd.1 for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:29:00.667 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1[39707]: 2024-09-17T09:29:00.303+0000 7fd7caafc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:29:00.667 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1[39707]: 2024-09-17T09:29:00.303+0000 7fd7caafc640 -1 osd.1 71 *** Got signal Terminated *** 2024-09-17T09:29:00.667 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:00 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1[39707]: 2024-09-17T09:29:00.303+0000 7fd7caafc640 -1 osd.1 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T09:29:06.167 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:05 smithi148 podman[70015]: 2024-09-17 09:29:05.861351448 +0000 UTC m=+5.696191177 container died 0557b456562f5e858baf554afb7db93cb415194f4e5679eddb10911c7370db9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b) 2024-09-17T09:29:06.667 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:06 smithi148 podman[70015]: 2024-09-17 09:29:06.341761047 +0000 UTC m=+6.176600808 container cleanup 0557b456562f5e858baf554afb7db93cb415194f4e5679eddb10911c7370db9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2) 2024-09-17T09:29:06.667 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:06 smithi148 bash[70015]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1 2024-09-17T09:29:06.668 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:06 smithi148 podman[70262]: 2024-09-17 09:29:06.501735169 +0000 UTC m=+0.866273980 container remove 0557b456562f5e858baf554afb7db93cb415194f4e5679eddb10911c7370db9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-17T09:29:08.023 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:07 smithi148 podman[70464]: 2024-09-17 09:29:07.671350158 +0000 UTC m=+0.022687510 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:29:08.023 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:07 smithi148 podman[70464]: 2024-09-17 09:29:07.791254802 +0000 UTC m=+0.142592145 container create b65c428c19200140bfa5e7e7dd0a674de7e6b1c0960880e31854958052aba1d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-17T09:29:08.023 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:08 smithi148 podman[70464]: 2024-09-17 09:29:08.01895911 +0000 UTC m=+0.370296460 container init b65c428c19200140bfa5e7e7dd0a674de7e6b1c0960880e31854958052aba1d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-17T09:29:08.023 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:08 smithi148 podman[70464]: 2024-09-17 09:29:08.022513258 +0000 UTC m=+0.373850604 container start b65c428c19200140bfa5e7e7dd0a674de7e6b1c0960880e31854958052aba1d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-17T09:29:08.307 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:08 smithi148 podman[70464]: 2024-09-17 09:29:08.119306502 +0000 UTC m=+0.470643859 container attach b65c428c19200140bfa5e7e7dd0a674de7e6b1c0960880e31854958052aba1d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-17T09:29:08.624 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 17 09:29:08 smithi148 podman[70464]: 2024-09-17 09:29:08.30674018 +0000 UTC m=+0.658077522 container died b65c428c19200140bfa5e7e7dd0a674de7e6b1c0960880e31854958052aba1d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-17T09:29:09.486 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.1.service' 2024-09-17T09:29:09.529 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:29:09.530 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-17T09:29:09.530 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-17T09:29:09.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.2 2024-09-17T09:29:09.858 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:09 smithi148 systemd[1]: Stopping Ceph osd.2 for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:29:10.147 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:09 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2[43223]: 2024-09-17T09:29:09.856+0000 7f53c1f54640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:29:10.147 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:09 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2[43223]: 2024-09-17T09:29:09.856+0000 7f53c1f54640 -1 osd.2 71 *** Got signal Terminated *** 2024-09-17T09:29:10.148 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:09 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2[43223]: 2024-09-17T09:29:09.856+0000 7f53c1f54640 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T09:29:15.667 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:15 smithi148 podman[70686]: 2024-09-17 09:29:15.263764977 +0000 UTC m=+5.502555903 container died 18327f211771206391bac6410ce716c95a4b60eb87acd351de15ad388da672b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T09:29:16.167 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:15 smithi148 podman[70686]: 2024-09-17 09:29:15.682084198 +0000 UTC m=+5.920875131 container cleanup 18327f211771206391bac6410ce716c95a4b60eb87acd351de15ad388da672b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-17T09:29:16.167 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:15 smithi148 bash[70686]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2 2024-09-17T09:29:16.167 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:15 smithi148 podman[70976]: 2024-09-17 09:29:15.891835042 +0000 UTC m=+0.624650683 container remove 18327f211771206391bac6410ce716c95a4b60eb87acd351de15ad388da672b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-17T09:29:16.596 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:16 smithi148 podman[71066]: 2024-09-17 09:29:16.292648949 +0000 UTC m=+0.109965787 container create 3cb107107c82f0d427b1b5300755dd13efc20366f032b868d145f7a9dcf62c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-17T09:29:16.596 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:16 smithi148 podman[71066]: 2024-09-17 09:29:16.198670321 +0000 UTC m=+0.015987123 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:29:16.596 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:16 smithi148 podman[71066]: 2024-09-17 09:29:16.512224121 +0000 UTC m=+0.329540923 container init 3cb107107c82f0d427b1b5300755dd13efc20366f032b868d145f7a9dcf62c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-17T09:29:16.596 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:16 smithi148 podman[71066]: 2024-09-17 09:29:16.515925735 +0000 UTC m=+0.333242529 container start 3cb107107c82f0d427b1b5300755dd13efc20366f032b868d145f7a9dcf62c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:29:16.596 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:16 smithi148 podman[71066]: 2024-09-17 09:29:16.595239802 +0000 UTC m=+0.412556609 container attach 3cb107107c82f0d427b1b5300755dd13efc20366f032b868d145f7a9dcf62c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-17T09:29:16.917 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 17 09:29:16 smithi148 podman[71066]: 2024-09-17 09:29:16.703305026 +0000 UTC m=+0.520621829 container died 3cb107107c82f0d427b1b5300755dd13efc20366f032b868d145f7a9dcf62c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b) 2024-09-17T09:29:17.467 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.2.service' 2024-09-17T09:29:17.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:29:17.508 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-17T09:29:17.508 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-17T09:29:17.508 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.3 2024-09-17T09:29:17.703 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:17 smithi148 systemd[1]: Stopping Ceph osd.3 for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:29:18.167 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:17 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3[46833]: 2024-09-17T09:29:17.868+0000 7fa84239f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:29:18.168 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:17 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3[46833]: 2024-09-17T09:29:17.868+0000 7fa84239f640 -1 osd.3 71 *** Got signal Terminated *** 2024-09-17T09:29:18.168 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:17 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3[46833]: 2024-09-17T09:29:17.868+0000 7fa84239f640 -1 osd.3 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T09:29:21.417 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:20 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3[46833]: 2024-09-17T09:29:20.961+0000 7fa83e9b3640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:55.485554+0000 front 2024-09-17T09:28:55.485697+0000 (oldest deadline 2024-09-17T09:29:20.185640+0000) 2024-09-17T09:29:22.274 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:21 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:21.777+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:55.486050+0000 front 2024-09-17T09:28:55.486242+0000 (oldest deadline 2024-09-17T09:29:20.786105+0000) 2024-09-17T09:29:22.417 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:21 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3[46833]: 2024-09-17T09:29:21.992+0000 7fa83e9b3640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:55.485554+0000 front 2024-09-17T09:28:55.485697+0000 (oldest deadline 2024-09-17T09:29:20.185640+0000) 2024-09-17T09:29:22.754 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:22 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:22.293+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:23.023 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:22 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:22.811+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:55.486050+0000 front 2024-09-17T09:28:55.486242+0000 (oldest deadline 2024-09-17T09:29:20.786105+0000) 2024-09-17T09:29:23.024 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:22 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:22.834+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:23.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:22 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:22.752+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:23.528 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:23 smithi148 podman[71169]: 2024-09-17 09:29:23.235948165 +0000 UTC m=+5.501405537 container died e784eb75d0462b51a4ea1b4631c690d13a6f9f7ba2ed735cb63e015153af83a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-17T09:29:23.771 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:23 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:23.273+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:23.854 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:23 smithi148 podman[71169]: 2024-09-17 09:29:23.676814468 +0000 UTC m=+5.942271873 container cleanup e784eb75d0462b51a4ea1b4631c690d13a6f9f7ba2ed735cb63e015153af83a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-17T09:29:23.854 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:23 smithi148 bash[71169]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3 2024-09-17T09:29:24.023 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:23 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:23.852+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:24.024 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:23 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:23.841+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:55.486050+0000 front 2024-09-17T09:28:55.486242+0000 (oldest deadline 2024-09-17T09:29:20.786105+0000) 2024-09-17T09:29:24.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:23 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:23.769+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:24.167 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:23 smithi148 podman[71198]: 2024-09-17 09:29:23.853155875 +0000 UTC m=+0.615717652 container remove e784eb75d0462b51a4ea1b4631c690d13a6f9f7ba2ed735cb63e015153af83a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD) 2024-09-17T09:29:24.573 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:24 smithi148 podman[71287]: 2024-09-17 09:29:24.187721815 +0000 UTC m=+0.015913800 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:29:24.573 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:24 smithi148 podman[71287]: 2024-09-17 09:29:24.286986137 +0000 UTC m=+0.115178118 container create bbafdcf2c3d689b952abcda22224b73fb33e4ab03fe4f5155fe4ee7623a01197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-17T09:29:24.573 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:24 smithi148 podman[71287]: 2024-09-17 09:29:24.498383756 +0000 UTC m=+0.326575739 container init bbafdcf2c3d689b952abcda22224b73fb33e4ab03fe4f5155fe4ee7623a01197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T09:29:24.573 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:24 smithi148 podman[71287]: 2024-09-17 09:29:24.501860603 +0000 UTC m=+0.330052577 container start bbafdcf2c3d689b952abcda22224b73fb33e4ab03fe4f5155fe4ee7623a01197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:29:24.573 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:24 smithi148 podman[71287]: 2024-09-17 09:29:24.572474662 +0000 UTC m=+0.400666643 container attach bbafdcf2c3d689b952abcda22224b73fb33e4ab03fe4f5155fe4ee7623a01197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b) 2024-09-17T09:29:24.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:24 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:24.320+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:24.917 INFO:journalctl@ceph.osd.3.smithi148.stdout:Sep 17 09:29:24 smithi148 podman[71287]: 2024-09-17 09:29:24.693819863 +0000 UTC m=+0.522011844 container died bbafdcf2c3d689b952abcda22224b73fb33e4ab03fe4f5155fe4ee7623a01197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:29:25.266 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.3.service' 2024-09-17T09:29:25.273 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:24 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:24.789+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:25.273 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:24 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:24.836+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:55.486050+0000 front 2024-09-17T09:28:55.486242+0000 (oldest deadline 2024-09-17T09:29:20.786105+0000) 2024-09-17T09:29:25.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:24 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:24.865+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:25.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:29:25.307 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-17T09:29:25.308 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-17T09:29:25.308 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.4 2024-09-17T09:29:25.637 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:25 smithi157 systemd[1]: Stopping Ceph osd.4 for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:29:25.637 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:25.314+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:26.023 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:25.819+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:26.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:25.813+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:26.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:25.813+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:26.024 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:25.635+0000 7f1c2abd7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:29:26.024 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:25.635+0000 7f1c2abd7640 -1 osd.4 71 *** Got signal Terminated *** 2024-09-17T09:29:26.024 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:25.635+0000 7f1c2abd7640 -1 osd.4 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T09:29:26.024 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:25 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:25.880+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:55.486050+0000 front 2024-09-17T09:28:55.486242+0000 (oldest deadline 2024-09-17T09:29:20.786105+0000) 2024-09-17T09:29:26.637 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:26.325+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:27.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:26.827+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:27.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:26.827+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:27.024 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:26.774+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:27.025 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:26.850+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:55.486050+0000 front 2024-09-17T09:28:55.486242+0000 (oldest deadline 2024-09-17T09:29:20.786105+0000) 2024-09-17T09:29:27.025 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:26 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:26.850+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:00.786455+0000 front 2024-09-17T09:29:00.786510+0000 (oldest deadline 2024-09-17T09:29:26.086442+0000) 2024-09-17T09:29:27.457 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:27.335+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:28.023 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:27.808+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:28.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:27.808+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:28.024 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:29:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:29:27.771Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.148:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.148:8765: connect: connection refused" 2024-09-17T09:29:28.024 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:29:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:29:27.773Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.148:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.148:8765: connect: connection refused" 2024-09-17T09:29:28.024 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:29:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:29:27.773Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.148:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.148:8765: connect: connection refused" 2024-09-17T09:29:28.024 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:29:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:29:27.774Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.148:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.148:8765: connect: connection refused" 2024-09-17T09:29:28.024 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:29:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:29:27.774Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.148:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.148:8765: connect: connection refused" 2024-09-17T09:29:28.025 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:27.820+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:55.486050+0000 front 2024-09-17T09:28:55.486242+0000 (oldest deadline 2024-09-17T09:29:20.786105+0000) 2024-09-17T09:29:28.025 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:27.820+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:00.786455+0000 front 2024-09-17T09:29:00.786510+0000 (oldest deadline 2024-09-17T09:29:26.086442+0000) 2024-09-17T09:29:28.025 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:27 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:27.816+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:28.576 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:28.307+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:28.848 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:28.794+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:55.486050+0000 front 2024-09-17T09:28:55.486242+0000 (oldest deadline 2024-09-17T09:29:20.786105+0000) 2024-09-17T09:29:28.849 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:28.794+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:00.786455+0000 front 2024-09-17T09:29:00.786510+0000 (oldest deadline 2024-09-17T09:29:26.086442+0000) 2024-09-17T09:29:28.849 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:28.782+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:28.849 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:28.782+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:28.849 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:28.790+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:28.849 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:28 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:28.790+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:29.523 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:29 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:29.328+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:30.023 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:29 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:29.767+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:55.486050+0000 front 2024-09-17T09:28:55.486242+0000 (oldest deadline 2024-09-17T09:29:20.786105+0000) 2024-09-17T09:29:30.024 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:29 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4[33757]: 2024-09-17T09:29:29.767+0000 7f1c231e3640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:00.786455+0000 front 2024-09-17T09:29:00.786510+0000 (oldest deadline 2024-09-17T09:29:26.086442+0000) 2024-09-17T09:29:30.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:29 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:29.805+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:30.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:29 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:29.805+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:30.024 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:29 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:29.758+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:30.024 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:29 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:29.758+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:30.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:30 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:30.330+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:30.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:30 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:30.330+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:31.270 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:30 smithi157 podman[63049]: 2024-09-17 09:29:30.957098656 +0000 UTC m=+5.438920058 container died f438f75a9020ab85293572c766a4b9061ebd15ffa403a076ed113b4dec990cad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-17T09:29:31.270 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:30 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:30.808+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:31.270 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:30 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:30.808+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:31.270 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:30 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:30.795+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:31.270 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:30 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:30.795+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:31.524 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:31 smithi157 podman[63049]: 2024-09-17 09:29:31.367165345 +0000 UTC m=+5.848986740 container cleanup f438f75a9020ab85293572c766a4b9061ebd15ffa403a076ed113b4dec990cad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-17T09:29:31.524 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:31 smithi157 bash[63049]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4 2024-09-17T09:29:31.524 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:31 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:31.312+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:31.524 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:31 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:31.312+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:31.930 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:31 smithi157 podman[63215]: 2024-09-17 09:29:31.601897857 +0000 UTC m=+0.635122919 container remove f438f75a9020ab85293572c766a4b9061ebd15ffa403a076ed113b4dec990cad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-17T09:29:31.930 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:31 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:31.836+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:31.930 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:31 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:31.836+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:31.930 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:31 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:31.760+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:31.930 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:31 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:31.760+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:32.189 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:31 smithi157 podman[63308]: 2024-09-17 09:29:31.83192764 +0000 UTC m=+0.016098086 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:29:32.189 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:31 smithi157 podman[63308]: 2024-09-17 09:29:31.944286712 +0000 UTC m=+0.128457147 container create c240d6fd0ccf79e907bc23bd5b375a5b53e89f1528ef17f1359a06d9a49712a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-17T09:29:32.189 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:32 smithi157 podman[63308]: 2024-09-17 09:29:32.188435603 +0000 UTC m=+0.372606038 container init c240d6fd0ccf79e907bc23bd5b375a5b53e89f1528ef17f1359a06d9a49712a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-17T09:29:32.523 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:32 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:32.342+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:32.524 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:32 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:32.342+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:32.524 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:32 smithi157 podman[63308]: 2024-09-17 09:29:32.191943941 +0000 UTC m=+0.376114385 container start c240d6fd0ccf79e907bc23bd5b375a5b53e89f1528ef17f1359a06d9a49712a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-17T09:29:32.524 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:32 smithi157 podman[63308]: 2024-09-17 09:29:32.271807897 +0000 UTC m=+0.455978344 container attach c240d6fd0ccf79e907bc23bd5b375a5b53e89f1528ef17f1359a06d9a49712a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-09-17T09:29:32.524 INFO:journalctl@ceph.osd.4.smithi157.stdout:Sep 17 09:29:32 smithi157 podman[63308]: 2024-09-17 09:29:32.375716267 +0000 UTC m=+0.559886703 container died c240d6fd0ccf79e907bc23bd5b375a5b53e89f1528ef17f1359a06d9a49712a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T09:29:32.917 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:32 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:32.788+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:32.918 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:32 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:32.788+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:32.918 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:32 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:32.757+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:32.918 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:32 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:32.757+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:32.941 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.4.service' 2024-09-17T09:29:32.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:29:32.982 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-17T09:29:32.983 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-17T09:29:32.983 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.5 2024-09-17T09:29:33.273 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:33 smithi157 systemd[1]: Stopping Ceph osd.5 for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:29:33.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:33 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:33.361+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:33.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:33 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:33.361+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:33.773 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:33 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:33.362+0000 7f7e37264640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:29:33.774 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:33 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:33.362+0000 7f7e37264640 -1 osd.5 71 *** Got signal Terminated *** 2024-09-17T09:29:33.774 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:33 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:33.362+0000 7f7e37264640 -1 osd.5 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T09:29:34.273 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:33 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:33.828+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:34.273 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:33 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:33.828+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:34.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:33 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:33.805+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:34.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:33 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:33.805+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:34.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:34 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:34.371+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:34.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:34 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:34.371+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:35.273 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:34 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:34.794+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:35.274 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:34 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:34.794+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:35.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:34 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:34.821+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:35.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:34 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:34.821+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:35.752 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:35 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:35.351+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:35.752 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:35 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:35.351+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:36.023 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:35 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:35.803+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:36.023 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:35 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:35.803+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:36.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:35 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:35.750+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:36.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:35 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:35.750+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:36.741 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:36 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:36.320+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:36.741 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:36 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:36.320+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:37.023 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:36 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:36.740+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:37.024 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:36 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:36.740+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:37.024 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:36 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:36.826+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:37.024 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:36 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:36.826+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:37.024 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:36 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:36.826+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:37.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:37 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:37.314+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:37.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:37 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:37.314+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:38.273 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:37 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:37.777+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.967985+0000 front 2024-09-17T09:28:56.968028+0000 (oldest deadline 2024-09-17T09:29:22.267851+0000) 2024-09-17T09:29:38.274 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:37 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:37.777+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.268300+0000 front 2024-09-17T09:29:02.268306+0000 (oldest deadline 2024-09-17T09:29:25.768286+0000) 2024-09-17T09:29:38.274 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:37 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5[37215]: 2024-09-17T09:29:37.777+0000 7f7e2f870640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.369734+0000 front 2024-09-17T09:29:13.369748+0000 (oldest deadline 2024-09-17T09:29:36.869695+0000) 2024-09-17T09:29:38.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:37 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:37.873+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:38.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:37 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:37.873+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:38.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:37 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:37.873+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:38.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:38 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:38.317+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:38.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:38 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:38.317+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:38.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:38 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:38.317+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:14.303596+0000 front 2024-09-17T09:29:14.303626+0000 (oldest deadline 2024-09-17T09:29:37.803505+0000) 2024-09-17T09:29:39.130 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:38 smithi157 podman[63414]: 2024-09-17 09:29:38.827349095 +0000 UTC m=+5.607798703 container died 20e4681fb2f576aaab97c946782c4ef28dea99f8c7c763448ca8a5151a5cf3d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:29:39.131 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:38 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:38.912+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:39.131 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:38 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:38.912+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:39.131 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:38 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:38.912+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:39.524 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:39 smithi157 podman[63414]: 2024-09-17 09:29:39.34513042 +0000 UTC m=+6.125580028 container cleanup 20e4681fb2f576aaab97c946782c4ef28dea99f8c7c763448ca8a5151a5cf3d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-17T09:29:39.524 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:39 smithi157 bash[63414]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5 2024-09-17T09:29:39.524 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:39 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:39.316+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:39.524 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:39 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:39.316+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:39.525 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:39 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:39.316+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:14.303596+0000 front 2024-09-17T09:29:14.303626+0000 (oldest deadline 2024-09-17T09:29:37.803505+0000) 2024-09-17T09:29:39.934 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:39 smithi157 podman[63580]: 2024-09-17 09:29:39.596729106 +0000 UTC m=+0.762073505 container remove 20e4681fb2f576aaab97c946782c4ef28dea99f8c7c763448ca8a5151a5cf3d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-17T09:29:40.259 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:39 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:39.948+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:40.259 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:39 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:39.948+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:40.259 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:39 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:39.948+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:40.259 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:39 smithi157 podman[63701]: 2024-09-17 09:29:39.837018261 +0000 UTC m=+0.022437376 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:29:40.259 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:39 smithi157 podman[63701]: 2024-09-17 09:29:39.939532533 +0000 UTC m=+0.124951707 container create caca0298acf2fbe0f8e3ae00820984fdab195bdb463dd81fe55b9c5bbce3cf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:29:40.259 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:40 smithi157 podman[63701]: 2024-09-17 09:29:40.175314564 +0000 UTC m=+0.360733683 container init caca0298acf2fbe0f8e3ae00820984fdab195bdb463dd81fe55b9c5bbce3cf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=reef-19e2a2b) 2024-09-17T09:29:40.259 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:40 smithi157 podman[63701]: 2024-09-17 09:29:40.178829397 +0000 UTC m=+0.364248513 container start caca0298acf2fbe0f8e3ae00820984fdab195bdb463dd81fe55b9c5bbce3cf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:29:40.259 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:40 smithi157 podman[63701]: 2024-09-17 09:29:40.258348001 +0000 UTC m=+0.443767124 container attach caca0298acf2fbe0f8e3ae00820984fdab195bdb463dd81fe55b9c5bbce3cf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:29:40.523 INFO:journalctl@ceph.osd.5.smithi157.stdout:Sep 17 09:29:40 smithi157 podman[63701]: 2024-09-17 09:29:40.370384267 +0000 UTC m=+0.555803382 container died caca0298acf2fbe0f8e3ae00820984fdab195bdb463dd81fe55b9c5bbce3cf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=reef-19e2a2b, maintainer=Guillaume Abrioux ) 2024-09-17T09:29:40.524 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:40 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:40.319+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:40.524 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:40 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:40.319+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:40.524 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:40 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:40.319+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:14.303596+0000 front 2024-09-17T09:29:14.303626+0000 (oldest deadline 2024-09-17T09:29:37.803505+0000) 2024-09-17T09:29:41.021 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:40 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:40.918+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:41.021 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:40 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:40.918+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:41.021 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:40 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:40.918+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:41.045 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.5.service' 2024-09-17T09:29:41.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:29:41.086 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-17T09:29:41.086 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-17T09:29:41.086 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.6 2024-09-17T09:29:41.273 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:41 smithi157 systemd[1]: Stopping Ceph osd.6 for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:29:41.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:41 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:41.314+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:41.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:41 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:41.314+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:41.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:41 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:41.314+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:14.303596+0000 front 2024-09-17T09:29:14.303626+0000 (oldest deadline 2024-09-17T09:29:37.803505+0000) 2024-09-17T09:29:41.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:41 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:41.432+0000 7fd516ade640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:29:41.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:41 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:41.432+0000 7fd516ade640 -1 osd.6 71 *** Got signal Terminated *** 2024-09-17T09:29:41.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:41 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:41.432+0000 7fd516ade640 -1 osd.6 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T09:29:41.917 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:29:41 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:29:41.625Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-09-17T09:29:41.918 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:29:41 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:29:41.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi157.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.157:8443: connect: connection refused" 2024-09-17T09:29:42.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:41 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:41.915+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:42.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:41 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:41.915+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:42.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:41 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:41.915+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:42.638 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:42 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:42.330+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:42.638 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:42 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:42.330+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:42.638 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:42 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:42.330+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:14.303596+0000 front 2024-09-17T09:29:14.303626+0000 (oldest deadline 2024-09-17T09:29:37.803505+0000) 2024-09-17T09:29:43.205 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:42 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:42.889+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:43.205 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:42 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:42.889+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:43.205 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:42 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:42.889+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:43.523 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:43.328+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:43.523 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:43.328+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:43.523 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:43.328+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:14.303596+0000 front 2024-09-17T09:29:14.303626+0000 (oldest deadline 2024-09-17T09:29:37.803505+0000) 2024-09-17T09:29:44.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:43.916+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:44.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:43.916+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:44.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:43 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:43.916+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:44.760 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:44 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:44.358+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:44.760 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:44 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:44.358+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:44.760 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:44 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:44.358+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:14.303596+0000 front 2024-09-17T09:29:14.303626+0000 (oldest deadline 2024-09-17T09:29:37.803505+0000) 2024-09-17T09:29:44.760 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:44 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:44.358+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:17.804116+0000 front 2024-09-17T09:29:17.804132+0000 (oldest deadline 2024-09-17T09:29:43.604521+0000) 2024-09-17T09:29:45.023 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:44 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:44.949+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:45.023 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:44 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:44.949+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:45.024 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:44 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:44.949+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:45.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:45.403+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:45.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:45.403+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:45.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:45.403+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:14.303596+0000 front 2024-09-17T09:29:14.303626+0000 (oldest deadline 2024-09-17T09:29:37.803505+0000) 2024-09-17T09:29:45.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:45.403+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:17.804116+0000 front 2024-09-17T09:29:17.804132+0000 (oldest deadline 2024-09-17T09:29:43.604521+0000) 2024-09-17T09:29:46.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:45.902+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:46.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:45.902+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:46.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:45 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:45.902+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:46.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:46.401+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:56.201476+0000 front 2024-09-17T09:28:56.201275+0000 (oldest deadline 2024-09-17T09:29:22.101189+0000) 2024-09-17T09:29:46.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:46.401+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:03.202364+0000 front 2024-09-17T09:29:03.202310+0000 (oldest deadline 2024-09-17T09:29:29.602688+0000) 2024-09-17T09:29:46.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:46.401+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:14.303596+0000 front 2024-09-17T09:29:14.303626+0000 (oldest deadline 2024-09-17T09:29:37.803505+0000) 2024-09-17T09:29:46.774 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6[40748]: 2024-09-17T09:29:46.401+0000 7fd50f0ea640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:17.804116+0000 front 2024-09-17T09:29:17.804132+0000 (oldest deadline 2024-09-17T09:29:43.604521+0000) 2024-09-17T09:29:47.224 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:46 smithi157 podman[63890]: 2024-09-17 09:29:46.958406327 +0000 UTC m=+5.638266443 container died 91eae7c18e856f59c68e3c3711cafbff4287d8f103ef3691ea9f9806ed791573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-17T09:29:47.225 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:46.871+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:47.225 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:46.871+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:47.225 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:46.871+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:47.225 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:46 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:46.871+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:21.983842+0000 front 2024-09-17T09:29:21.983830+0000 (oldest deadline 2024-09-17T09:29:46.584406+0000) 2024-09-17T09:29:47.506 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:47 smithi157 podman[63890]: 2024-09-17 09:29:47.373046707 +0000 UTC m=+6.052906821 container cleanup 91eae7c18e856f59c68e3c3711cafbff4287d8f103ef3691ea9f9806ed791573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6, ceph=True, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-17T09:29:47.507 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:47 smithi157 bash[63890]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6 2024-09-17T09:29:47.773 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:47 smithi157 podman[64304]: 2024-09-17 09:29:47.608408604 +0000 UTC m=+0.643051342 container remove 91eae7c18e856f59c68e3c3711cafbff4287d8f103ef3691ea9f9806ed791573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=reef-19e2a2b, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T09:29:48.273 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:48 smithi157 podman[64332]: 2024-09-17 09:29:48.027628483 +0000 UTC m=+0.627511721 container died 2e97983669a2a03e821f404289a33efb466fff19a3fcc7bdb2015b40d868d5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=dazzling_wilbur, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-17T09:29:48.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:47 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:47.876+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:48.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:47 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:47.876+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:48.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:47 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:47.876+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:48.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:47 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:47.876+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:21.983842+0000 front 2024-09-17T09:29:21.983830+0000 (oldest deadline 2024-09-17T09:29:46.584406+0000) 2024-09-17T09:29:48.930 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:48 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:48.887+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:48.930 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:48 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:48.887+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:48.930 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:48 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:48.887+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:48.930 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:48 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:48.887+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:21.983842+0000 front 2024-09-17T09:29:21.983830+0000 (oldest deadline 2024-09-17T09:29:46.584406+0000) 2024-09-17T09:29:49.630 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:49 smithi157 podman[64465]: 2024-09-17 09:29:49.261936589 +0000 UTC m=+0.016314481 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:29:49.630 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:49 smithi157 podman[64465]: 2024-09-17 09:29:49.393283251 +0000 UTC m=+0.147661143 container create 5e6abf0074a65effe83eebe3bbe7370e26d221910a0f9ad0228fbe008b0e668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-17T09:29:49.630 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:49 smithi157 podman[64465]: 2024-09-17 09:29:49.62882625 +0000 UTC m=+0.383204145 container init 5e6abf0074a65effe83eebe3bbe7370e26d221910a0f9ad0228fbe008b0e668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-17T09:29:49.906 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:49 smithi157 podman[64465]: 2024-09-17 09:29:49.63256401 +0000 UTC m=+0.386941903 container start 5e6abf0074a65effe83eebe3bbe7370e26d221910a0f9ad0228fbe008b0e668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-17T09:29:49.906 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:49 smithi157 podman[64465]: 2024-09-17 09:29:49.729131911 +0000 UTC m=+0.483509826 container attach 5e6abf0074a65effe83eebe3bbe7370e26d221910a0f9ad0228fbe008b0e668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:29:50.209 INFO:journalctl@ceph.osd.6.smithi157.stdout:Sep 17 09:29:49 smithi157 podman[64465]: 2024-09-17 09:29:49.959802838 +0000 UTC m=+0.714180738 container died 5e6abf0074a65effe83eebe3bbe7370e26d221910a0f9ad0228fbe008b0e668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:29:50.209 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:49 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:49.904+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:50.209 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:49 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:49.904+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:50.209 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:49 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:49.904+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:50.210 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:49 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:49.904+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:21.983842+0000 front 2024-09-17T09:29:21.983830+0000 (oldest deadline 2024-09-17T09:29:46.584406+0000) 2024-09-17T09:29:50.735 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.6.service' 2024-09-17T09:29:50.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:29:50.777 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-17T09:29:50.778 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-17T09:29:50.778 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.7 2024-09-17T09:29:50.953 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:50 smithi157 systemd[1]: Stopping Ceph osd.7 for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:29:51.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:50 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:50.952+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:51.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:50 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:50.952+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:51.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:50 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:50.952+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:51.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:50 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:50.952+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:21.983842+0000 front 2024-09-17T09:29:21.983830+0000 (oldest deadline 2024-09-17T09:29:46.584406+0000) 2024-09-17T09:29:51.773 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:51 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:51.438+0000 7efc6af08640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:29:51.773 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:51 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:51.439+0000 7efc6af08640 -1 osd.7 71 *** Got signal Terminated *** 2024-09-17T09:29:51.774 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:51 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:51.439+0000 7efc6af08640 -1 osd.7 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T09:29:51.917 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:29:51 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:29:51.625Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi157.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.157:8443: connect: connection refused" 2024-09-17T09:29:51.918 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:29:51 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:29:51.625Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-09-17T09:29:51.918 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:29:51 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:29:51.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi157.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.157:8443: connect: connection refused" 2024-09-17T09:29:52.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:51 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:51.909+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:52.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:51 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:51.909+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:52.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:51 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:51.909+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:52.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:51 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:51.909+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:21.983842+0000 front 2024-09-17T09:29:21.983830+0000 (oldest deadline 2024-09-17T09:29:46.584406+0000) 2024-09-17T09:29:53.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:52 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:52.944+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:53.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:52 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:52.944+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:53.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:52 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:52.944+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:53.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:52 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:52.944+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:21.983842+0000 front 2024-09-17T09:29:21.983830+0000 (oldest deadline 2024-09-17T09:29:46.584406+0000) 2024-09-17T09:29:54.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:53 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:53.991+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:54.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:53 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:53.991+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:54.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:53 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:53.991+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:54.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:53 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:53.991+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:21.983842+0000 front 2024-09-17T09:29:21.983830+0000 (oldest deadline 2024-09-17T09:29:46.584406+0000) 2024-09-17T09:29:54.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:53 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:53.991+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.157:6804 osd.4 since back 2024-09-17T09:29:29.485185+0000 front 2024-09-17T09:29:29.485172+0000 (oldest deadline 2024-09-17T09:29:52.985185+0000) 2024-09-17T09:29:55.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:54 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:54.943+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:55.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:54 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:54.943+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:55.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:54 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:54.943+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:55.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:54 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:54.943+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:21.983842+0000 front 2024-09-17T09:29:21.983830+0000 (oldest deadline 2024-09-17T09:29:46.584406+0000) 2024-09-17T09:29:55.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:54 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:54.943+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.157:6804 osd.4 since back 2024-09-17T09:29:29.485185+0000 front 2024-09-17T09:29:29.485172+0000 (oldest deadline 2024-09-17T09:29:52.985185+0000) 2024-09-17T09:29:56.273 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:55 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:55.916+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6806 osd.0 since back 2024-09-17T09:28:52.080669+0000 front 2024-09-17T09:28:52.080611+0000 (oldest deadline 2024-09-17T09:29:22.681191+0000) 2024-09-17T09:29:56.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:55 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:55.916+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6814 osd.1 since back 2024-09-17T09:29:02.681835+0000 front 2024-09-17T09:29:02.681732+0000 (oldest deadline 2024-09-17T09:29:27.981705+0000) 2024-09-17T09:29:56.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:55 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:55.916+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6822 osd.2 since back 2024-09-17T09:29:13.183041+0000 front 2024-09-17T09:29:13.183050+0000 (oldest deadline 2024-09-17T09:29:36.682927+0000) 2024-09-17T09:29:56.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:55 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:55.916+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.148:6830 osd.3 since back 2024-09-17T09:29:21.983842+0000 front 2024-09-17T09:29:21.983830+0000 (oldest deadline 2024-09-17T09:29:46.584406+0000) 2024-09-17T09:29:56.274 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:55 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7[44668]: 2024-09-17T09:29:55.916+0000 7efc63514640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.157:6804 osd.4 since back 2024-09-17T09:29:29.485185+0000 front 2024-09-17T09:29:29.485172+0000 (oldest deadline 2024-09-17T09:29:52.985185+0000) 2024-09-17T09:29:57.190 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:56 smithi157 podman[64640]: 2024-09-17 09:29:56.920900249 +0000 UTC m=+5.576619713 container died a97be771a25d04218073594fbf3eca87e49946ee50aa3afdc2bd8c2191a45c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-17T09:29:57.523 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:57 smithi157 podman[64640]: 2024-09-17 09:29:57.347120034 +0000 UTC m=+6.002839533 container cleanup a97be771a25d04218073594fbf3eca87e49946ee50aa3afdc2bd8c2191a45c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-17T09:29:57.524 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:57 smithi157 bash[64640]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7 2024-09-17T09:29:57.957 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:57 smithi157 podman[64654]: 2024-09-17 09:29:57.531525165 +0000 UTC m=+0.609594686 container remove a97be771a25d04218073594fbf3eca87e49946ee50aa3afdc2bd8c2191a45c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7, org.label-schema.vendor=CentOS, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T09:29:58.269 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:57 smithi157 podman[64745]: 2024-09-17 09:29:57.859555084 +0000 UTC m=+0.015845282 image pull 980a0bfcd684674f25c08d857ad8de3b1c8ae738870529edbf9aa463832c8d89 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9 2024-09-17T09:29:58.269 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:57 smithi157 podman[64745]: 2024-09-17 09:29:57.95717513 +0000 UTC m=+0.113465314 container create 18c0955b645d817651b8c825e8dec6b03f080bffcfae555408b0c6bf8af81544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-17T09:29:58.269 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:58 smithi157 podman[64745]: 2024-09-17 09:29:58.201334324 +0000 UTC m=+0.357624521 container init 18c0955b645d817651b8c825e8dec6b03f080bffcfae555408b0c6bf8af81544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=reef-19e2a2b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-17T09:29:58.269 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:58 smithi157 podman[64745]: 2024-09-17 09:29:58.204835928 +0000 UTC m=+0.361126113 container start 18c0955b645d817651b8c825e8dec6b03f080bffcfae555408b0c6bf8af81544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-17T09:29:58.269 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:58 smithi157 podman[64745]: 2024-09-17 09:29:58.267919438 +0000 UTC m=+0.424209624 container attach 18c0955b645d817651b8c825e8dec6b03f080bffcfae555408b0c6bf8af81544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-17T09:29:58.523 INFO:journalctl@ceph.osd.7.smithi157.stdout:Sep 17 09:29:58 smithi157 podman[64745]: 2024-09-17 09:29:58.386819738 +0000 UTC m=+0.543109923 container died 18c0955b645d817651b8c825e8dec6b03f080bffcfae555408b0c6bf8af81544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b) 2024-09-17T09:29:59.094 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@osd.7.service' 2024-09-17T09:29:59.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:29:59.134 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-17T09:29:59.134 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-17T09:29:59.134 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-17T09:29:59.417 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:29:59 smithi148 systemd[1]: Stopping Ceph rgw.foo.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:29:59.917 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:29:59 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-rgw-foo-a[50547]: 2024-09-17T09:29:59.442+0000 7f2b67430640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T09:29:59.917 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:29:59 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-rgw-foo-a[50547]: 2024-09-17T09:29:59.442+0000 7f2b6ac9b880 -1 shutting down 2024-09-17T09:30:01.700 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:01 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:30:01.625Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi157.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.157:8443: connect: connection refused" 2024-09-17T09:30:01.701 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:01 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:30:01.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-09-17T09:30:01.701 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:01 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:30:01.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi157.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.157:8443: connect: connection refused" 2024-09-17T09:30:09.788 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:30:09 smithi148 bash[72373]: time="2024-09-17T09:30:09Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-17T09:30:09.789 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:30:09 smithi148 podman[72373]: 2024-09-17 09:30:09.466716783 +0000 UTC m=+10.130401415 container died a7ed1a8a3f967f1e7793d1e5382b1f6eaa1804cb899b54cfc430158a6e4b9c17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-rgw-foo-a, RELEASE=reef-19e2a2b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True) 2024-09-17T09:30:10.079 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:30:09 smithi148 podman[72373]: 2024-09-17 09:30:09.877489162 +0000 UTC m=+10.541173782 container cleanup a7ed1a8a3f967f1e7793d1e5382b1f6eaa1804cb899b54cfc430158a6e4b9c17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=reef-19e2a2b) 2024-09-17T09:30:10.079 INFO:journalctl@ceph.rgw.foo.a.smithi148.stdout:Sep 17 09:30:09 smithi148 bash[72373]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-17T09:30:10.238 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-17T09:30:10.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:30:10.280 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-17T09:30:10.280 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-17T09:30:10.281 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@prometheus.a 2024-09-17T09:30:10.631 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:10 smithi157 systemd[1]: Stopping Ceph prometheus.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:30:11.274 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.100Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-17T09:30:11.274 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.101Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-17T09:30:11.274 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.101Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-17T09:30:11.274 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.101Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-17T09:30:11.274 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.101Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-17T09:30:11.274 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.101Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-17T09:30:11.274 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.101Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-17T09:30:11.274 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.101Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-17T09:30:11.274 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.101Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-17T09:30:11.274 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.101Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-17T09:30:11.275 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.101Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-17T09:30:11.275 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a[60978]: ts=2024-09-17T09:30:11.101Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-17T09:30:11.275 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 podman[65652]: 2024-09-17 09:30:11.11127661 +0000 UTC m=+0.118889077 container died c87d17afe318f05fa60790b07dee95f52048b47489905e007dc53e848edfd0fc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:30:11.736 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 podman[65652]: 2024-09-17 09:30:11.509123664 +0000 UTC m=+0.516736125 container cleanup c87d17afe318f05fa60790b07dee95f52048b47489905e007dc53e848edfd0fc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-17T09:30:11.736 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Sep 17 09:30:11 smithi157 bash[65652]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-prometheus-a 2024-09-17T09:30:11.886 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-17T09:30:11.917 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:11 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:30:11.626Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi157.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.157:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-09-17T09:30:11.917 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:11 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:30:11.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-09-17T09:30:11.917 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:11 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:30:11.627Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi157.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.157:8443: connect: connection refused" 2024-09-17T09:30:11.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T09:30:11.923 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-17T09:30:11.923 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T09:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:30:14.109 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:13 smithi148 systemd[1]: Stopping Ceph alertmanager.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:30:14.417 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:14 smithi148 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a[67123]: ts=2024-09-17T09:30:14.108Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-17T09:30:14.417 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:14 smithi148 podman[73599]: 2024-09-17 09:30:14.128611689 +0000 UTC m=+0.116820941 container died c5067a4f15d8779b607189c12be664598ee6c365db6df2206c31c122087b29ed (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:30:14.843 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:14 smithi148 podman[73599]: 2024-09-17 09:30:14.649620742 +0000 UTC m=+0.637829996 container cleanup c5067a4f15d8779b607189c12be664598ee6c365db6df2206c31c122087b29ed (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:30:14.843 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:14 smithi148 bash[73599]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a 2024-09-17T09:30:15.107 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:14 smithi148 podman[73610]: 2024-09-17 09:30:14.842376876 +0000 UTC m=+0.708767975 container remove c5067a4f15d8779b607189c12be664598ee6c365db6df2206c31c122087b29ed (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-17T09:30:15.107 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:14 smithi148 podman[73610]: 2024-09-17 09:30:14.91795042 +0000 UTC m=+0.784341522 volume remove abb5e38a735b1a914b7313c41207475975c1fe52fe30b81d911e2c5aea58ab7f 2024-09-17T09:30:15.417 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:30:15 smithi148 systemd[1]: Stopping Ceph node-exporter.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:30:15.418 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:15 smithi148 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-17T09:30:15.418 INFO:journalctl@ceph.alertmanager.a.smithi148.stdout:Sep 17 09:30:15 smithi148 systemd[1]: Stopped Ceph alertmanager.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:30:15.822 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:30:15 smithi148 podman[73720]: 2024-09-17 09:30:15.565126857 +0000 UTC m=+0.109353114 container died e7344f84d46e53b3d20a73ec73cd61febdd92e7913a34f7ee69ad566ade81300 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:30:16.167 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:30:16 smithi148 podman[73720]: 2024-09-17 09:30:16.02126184 +0000 UTC m=+0.565488136 container cleanup e7344f84d46e53b3d20a73ec73cd61febdd92e7913a34f7ee69ad566ade81300 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:30:16.167 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:30:16 smithi148 bash[73720]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a 2024-09-17T09:30:16.597 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:30:16 smithi148 podman[73732]: 2024-09-17 09:30:16.280943246 +0000 UTC m=+0.719866337 container remove e7344f84d46e53b3d20a73ec73cd61febdd92e7913a34f7ee69ad566ade81300 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-17T09:30:16.597 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:30:16 smithi148 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-17T09:30:16.597 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:30:16 smithi148 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-17T09:30:16.597 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:30:16 smithi148 systemd[1]: Stopped Ceph node-exporter.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:30:16.598 INFO:journalctl@ceph.node-exporter.a.smithi148.stdout:Sep 17 09:30:16 smithi148 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 1.922s CPU time. 2024-09-17T09:30:17.089 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T09:30:17.243 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:30:18.931 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:18 smithi157 systemd[1]: Stopping Ceph iscsi.iscsi.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:30:19.277 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:18 smithi157 ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a[47349]: debug Shutdown received 2024-09-17T09:30:29.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:28 smithi157 bash[66063]: time="2024-09-17T09:30:28Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-17T09:30:29.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:28 smithi157 podman[66063]: 2024-09-17 09:30:28.951607285 +0000 UTC m=+10.133611501 container died c1dd8f7936ffb9b0667e3b7f2da26e21327ad106c3dde9c9e707b4e3a7d991bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-19e2a2b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-17T09:30:29.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:29 smithi157 podman[66063]: 2024-09-17 09:30:29.412363324 +0000 UTC m=+10.594367503 container cleanup c1dd8f7936ffb9b0667e3b7f2da26e21327ad106c3dde9c9e707b4e3a7d991bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-17T09:30:29.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:29 smithi157 bash[66063]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a 2024-09-17T09:30:29.953 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:29 smithi157 podman[66076]: 2024-09-17 09:30:29.69697662 +0000 UTC m=+0.741983723 container remove c1dd8f7936ffb9b0667e3b7f2da26e21327ad106c3dde9c9e707b4e3a7d991bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_CLEAN=True, RELEASE=reef-19e2a2b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-17T09:30:29.953 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:29 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-17T09:30:30.209 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:29 smithi157 podman[66120]: 2024-09-17 09:30:29.952130345 +0000 UTC m=+0.115726033 container died 46418b5f17c12590c5c497601d31f504b7598fc2a692a322d4ef25a5504a0de5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-17T09:30:30.502 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:30 smithi157 podman[66120]: 2024-09-17 09:30:30.282917457 +0000 UTC m=+0.446513137 container cleanup 46418b5f17c12590c5c497601d31f504b7598fc2a692a322d4ef25a5504a0de5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-19e2a2b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-17T09:30:30.502 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:30 smithi157 bash[66120]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-17T09:30:30.502 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:30 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-17T09:30:30.502 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:30 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 47272 (conmon) remains running after unit stopped. 2024-09-17T09:30:30.502 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:30 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 66132 (podman) remains running after unit stopped. 2024-09-17T09:30:30.502 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:30 smithi157 systemd[1]: Stopped Ceph iscsi.iscsi.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:30:30.502 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:30 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Consumed 2.457s CPU time. 2024-09-17T09:30:30.773 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:30:30 smithi157 systemd[1]: Stopping Ceph grafana.a for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:30:30.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Sep 17 09:30:30 smithi157 podman[66132]: 2024-09-17 09:30:30.50092761 +0000 UTC m=+0.547311477 container remove 46418b5f17c12590c5c497601d31f504b7598fc2a692a322d4ef25a5504a0de5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b7810862ffedc21315a6c67a84878d4bbd16155279a41a670f6f6be7488da9, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=reef-19e2a2b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-17T09:30:31.214 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:30:30 smithi157 podman[66250]: 2024-09-17 09:30:30.947847186 +0000 UTC m=+0.116231596 container died 3c855b0d97f6c38ad127f86c8ada1fa0a232855cb73ca9f238dea8a0fea2bd4f (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, name=ubi8, com.redhat.component=ubi8-container, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, description=Ceph Grafana Container, io.openshift.expose-services=, architecture=x86_64, io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8, io.buildah.version=1.32.0, version=8.5, vendor=Red Hat, Inc., distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-09-17T09:30:31.523 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:30:31 smithi157 podman[66250]: 2024-09-17 09:30:31.37872441 +0000 UTC m=+0.547108821 container cleanup 3c855b0d97f6c38ad127f86c8ada1fa0a232855cb73ca9f238dea8a0fea2bd4f (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., io.openshift.expose-services=, maintainer=Paul Cuzner , vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, architecture=x86_64, io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8, io.buildah.version=1.32.0, version=8.5, release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, name=ubi8, description=Ceph Grafana Container) 2024-09-17T09:30:31.523 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:30:31 smithi157 bash[66250]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a 2024-09-17T09:30:31.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:30:31 smithi157 podman[66264]: 2024-09-17 09:30:31.588285993 +0000 UTC m=+0.638745450 container remove 3c855b0d97f6c38ad127f86c8ada1fa0a232855cb73ca9f238dea8a0fea2bd4f (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-grafana-a, com.redhat.component=ubi8-container, distribution-scope=public, release=239.1651231664, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, name=ubi8, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, io.buildah.version=1.32.0, maintainer=Paul Cuzner , version=8.5, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-09-17T09:30:31.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:30:31 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-17T09:30:31.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:30:31 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-17T09:30:31.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:30:31 smithi157 systemd[1]: Stopped Ceph grafana.a for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:30:31.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Sep 17 09:30:31 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@grafana.a.service: Consumed 5.034s CPU time. 2024-09-17T09:30:32.261 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:30:32 smithi157 systemd[1]: Stopping Ceph node-exporter.b for ff0bb856-74d5-11ef-bceb-c7b262605968... 2024-09-17T09:30:32.523 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:30:32 smithi157 podman[66382]: 2024-09-17 09:30:32.26031673 +0000 UTC m=+0.089979311 container died 55850ab76da2f24467764b284aed2a1509f0fd22584381f4909276c33a516519 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:30:32.834 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:30:32 smithi157 podman[66382]: 2024-09-17 09:30:32.641194091 +0000 UTC m=+0.470856659 container cleanup 55850ab76da2f24467764b284aed2a1509f0fd22584381f4909276c33a516519 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:30:32.834 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:30:32 smithi157 bash[66382]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b 2024-09-17T09:30:33.209 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:30:32 smithi157 podman[66394]: 2024-09-17 09:30:32.833756558 +0000 UTC m=+0.573411966 container remove 55850ab76da2f24467764b284aed2a1509f0fd22584381f4909276c33a516519 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ff0bb856-74d5-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-17T09:30:33.209 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:30:32 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-17T09:30:33.209 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:30:32 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-17T09:30:33.209 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:30:32 smithi157 systemd[1]: Stopped Ceph node-exporter.b for ff0bb856-74d5-11ef-bceb-c7b262605968. 2024-09-17T09:30:33.210 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Sep 17 09:30:32 smithi157 systemd[1]: ceph-ff0bb856-74d5-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 1.869s CPU time. 2024-09-17T09:30:33.645 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:30:33.678 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T09:30:33.706 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-17T09:30:33.707 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908296/remote/smithi148/crash 2024-09-17T09:30:33.708 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/crash -- . 2024-09-17T09:30:33.918 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T09:30:33.918 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2024-09-17T09:30:33.920 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908296/remote/smithi157/crash 2024-09-17T09:30:33.921 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/crash -- . 2024-09-17T09:30:33.967 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T09:30:33.967 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2024-09-17T09:30:33.968 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-17T09:30:33.968 DEBUG:teuthology.orchestra.run.smithi148:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v CEPHADM_STRAY_DAEMON | egrep -v CEPHADM_FAILED_DAEMON | egrep -v CEPHADM_AGENT_DOWN | head -n 1 2024-09-17T09:30:33.999 INFO:teuthology.orchestra.run.smithi148.stderr:grep: /var/log/ceph/ff0bb856-74d5-11ef-bceb-c7b262605968/ceph.log: No such file or directory 2024-09-17T09:30:34.001 INFO:tasks.cephadm:Compressing logs... 2024-09-17T09:30:34.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T09:30:34.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T09:30:34.069 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T09:30:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-17T09:30:35.633 INFO:tasks.cephadm:Archiving logs... 2024-09-17T09:30:35.633 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908296/remote/smithi148/log 2024-09-17T09:30:35.634 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T09:30:35.924 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908296/remote/smithi157/log 2024-09-17T09:30:35.925 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T09:30:36.103 INFO:tasks.cephadm:Removing cluster... 2024-09-17T09:30:36.104 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 --force 2024-09-17T09:30:36.242 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:30:36.369 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff0bb856-74d5-11ef-bceb-c7b262605968 --force 2024-09-17T09:30:36.521 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: ff0bb856-74d5-11ef-bceb-c7b262605968 2024-09-17T09:30:36.658 INFO:tasks.cephadm:Removing cephadm ... 2024-09-17T09:30:36.658 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T09:30:36.675 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T09:30:36.693 INFO:tasks.cephadm:Teardown complete 2024-09-17T09:30:36.694 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-09-17T09:30:36.706 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi148:/dev/vg_nvme/lv_1... 2024-09-17T09:30:36.706 DEBUG:teuthology.orchestra.run.smithi148:> sudo nvme disconnect -n lv_1 2024-09-17T09:30:36.871 INFO:teuthology.orchestra.run.smithi148.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-17T09:30:36.873 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi148:/dev/vg_nvme/lv_2... 2024-09-17T09:30:36.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo nvme disconnect -n lv_2 2024-09-17T09:30:37.020 INFO:teuthology.orchestra.run.smithi148.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-17T09:30:37.021 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi148:/dev/vg_nvme/lv_3... 2024-09-17T09:30:37.022 DEBUG:teuthology.orchestra.run.smithi148:> sudo nvme disconnect -n lv_3 2024-09-17T09:30:37.202 INFO:teuthology.orchestra.run.smithi148.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-17T09:30:37.204 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi148:/dev/vg_nvme/lv_4... 2024-09-17T09:30:37.204 DEBUG:teuthology.orchestra.run.smithi148:> sudo nvme disconnect -n lv_4 2024-09-17T09:30:37.359 INFO:teuthology.orchestra.run.smithi148.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-17T09:30:37.361 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-17T09:30:37.361 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/scratch_devs 2024-09-17T09:30:37.385 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi157:/dev/vg_nvme/lv_1... 2024-09-17T09:30:37.386 DEBUG:teuthology.orchestra.run.smithi157:> sudo nvme disconnect -n lv_1 2024-09-17T09:30:37.529 INFO:teuthology.orchestra.run.smithi157.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-17T09:30:37.531 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi157:/dev/vg_nvme/lv_2... 2024-09-17T09:30:37.531 DEBUG:teuthology.orchestra.run.smithi157:> sudo nvme disconnect -n lv_2 2024-09-17T09:30:37.687 INFO:teuthology.orchestra.run.smithi157.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-17T09:30:37.689 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi157:/dev/vg_nvme/lv_3... 2024-09-17T09:30:37.689 DEBUG:teuthology.orchestra.run.smithi157:> sudo nvme disconnect -n lv_3 2024-09-17T09:30:37.834 INFO:teuthology.orchestra.run.smithi157.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-17T09:30:37.836 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi157:/dev/vg_nvme/lv_4... 2024-09-17T09:30:37.836 DEBUG:teuthology.orchestra.run.smithi157:> sudo nvme disconnect -n lv_4 2024-09-17T09:30:37.975 INFO:teuthology.orchestra.run.smithi157.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-17T09:30:37.977 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-17T09:30:37.977 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/scratch_devs 2024-09-17T09:30:38.004 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T09:30:38.014 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T09:30:38.014 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T09:30:38.017 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T09:30:38.029 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-17T09:30:38.033 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T09:30:38.033 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-17T09:30:38.033 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 -285us[ -255us] +/- 40ms 2024-09-17T09:30:38.034 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -940us[ -911us] +/- 60ms 2024-09-17T09:30:38.034 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +1803us[+1833us] +/- 80ms 2024-09-17T09:30:38.034 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:30:38.059 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-09-17T09:30:38.064 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T09:30:38.064 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-09-17T09:30:38.064 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 -287us[ -303us] +/- 40ms 2024-09-17T09:30:38.064 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -937us[ -937us] +/- 60ms 2024-09-17T09:30:38.064 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +1814us[+1798us] +/- 80ms 2024-09-17T09:30:38.064 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T09:30:38.065 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T09:30:38.109 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T09:30:38.110 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T09:30:38.123 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T09:30:38.195 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T09:30:38.266 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T09:30:38.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:30:38.294 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T09:30:38.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:30:38.323 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T09:30:38.332 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T09:30:38.343 INFO:teuthology.task.internal:Duration was 884.570537 seconds 2024-09-17T09:30:38.343 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T09:30:38.354 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T09:30:38.355 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T09:30:38.357 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T09:30:38.393 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T09:30:38.402 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T09:30:38.562 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T09:30:38.562 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-09-17T09:30:38.563 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T09:30:38.583 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-09-17T09:30:38.584 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T09:30:38.625 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T09:30:38.625 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T09:30:38.628 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T09:30:38.693 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T09:30:38.694 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T09:30:38.696 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T09:30:38.963 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T09:30:38.972 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T09:30:38.973 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T09:30:38.997 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T09:30:39.023 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T09:30:39.034 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T09:30:39.040 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T09:30:39.061 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-09-17T09:30:39.086 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-09-17T09:30:39.121 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T09:30:39.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:30:39.140 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T09:30:39.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T09:30:39.176 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T09:30:39.186 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T09:30:39.187 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908296/remote/smithi148 2024-09-17T09:30:39.187 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T09:30:39.227 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_22:08:02-orch-reef-distro-default-smithi/7908296/remote/smithi157 2024-09-17T09:30:39.227 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T09:30:39.263 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T09:30:39.263 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T09:30:39.266 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T09:30:39.312 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T09:30:39.325 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T09:30:39.325 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T09:30:39.393 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T09:30:39.393 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T09:30:39.396 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T09:30:39.408 INFO:teuthology.orchestra.run.smithi148.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 09:30 /home/ubuntu/cephtest 2024-09-17T09:30:39.411 INFO:teuthology.orchestra.run.smithi157.stdout: 265836 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 09:30 /home/ubuntu/cephtest 2024-09-17T09:30:39.412 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T09:30:39.491 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T09:30:39.527 INFO:teuthology.run:Summary data: description: orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} duration: 884.5705370903015 owner: scheduled_teuthology@teuthology success: true 2024-09-17T09:30:39.528 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T09:30:39.639 INFO:teuthology.run:pass