2024-09-16T10:19:19.620 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T10:19:19.635 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T10:19:19.783 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906653 branch: reef description: rados/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906653' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - MON_DOWN - mons down - mon down - out of quorum - CEPHADM_STRAY_DAEMON - CEPHADM_FAILED_DAEMON - CEPHADM_AGENT_DOWN log-only-match: - CEPHADM_ sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: reef sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 owner: scheduled_teuthology@teuthology priority: 931 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 1390 sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 sleep_before_teardown: 0 subset: 5319/100000 suite: rados suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 targets: smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL7FMhSKNaxFEv9YQRcO2wCZj7vJvAzqQ2IMVsCjOqFmZivEdppJSGp7KsNGp0j4fbGKluhPBKn1TGN1UmBC5ME= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3f+RnIH8xelCs3IZQQUwGXlxFVkL5QZF1TyUawGQGcE8yrwSxhKaY0mmvt6W8KV9LNnIzY/0FwGi3PzKg3DLk= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_22:00:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T10:19:19.784 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa; will attempt to use it 2024-09-16T10:19:19.784 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa/tasks 2024-09-16T10:19:19.785 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T10:19:19.787 INFO:teuthology.task.internal:Checking packages... 2024-09-16T10:19:19.810 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8' 2024-09-16T10:19:19.810 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T10:19:19.811 INFO:teuthology.packaging:ref: None 2024-09-16T10:19:19.811 INFO:teuthology.packaging:tag: None 2024-09-16T10:19:19.811 INFO:teuthology.packaging:branch: reef 2024-09-16T10:19:19.811 INFO:teuthology.packaging:sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T10:19:19.811 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-16T10:19:20.101 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-853.gae7f3733 2024-09-16T10:19:20.102 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T10:19:20.109 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T10:19:20.109 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T10:19:20.118 INFO:teuthology.task.internal:Saving configuration 2024-09-16T10:19:20.129 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T10:19:20.139 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T10:19:20.165 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906653', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 10:14:03.945712', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL7FMhSKNaxFEv9YQRcO2wCZj7vJvAzqQ2IMVsCjOqFmZivEdppJSGp7KsNGp0j4fbGKluhPBKn1TGN1UmBC5ME='} 2024-09-16T10:19:20.189 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906653', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 10:14:03.947162', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3f+RnIH8xelCs3IZQQUwGXlxFVkL5QZF1TyUawGQGcE8yrwSxhKaY0mmvt6W8KV9LNnIzY/0FwGi3PzKg3DLk='} 2024-09-16T10:19:20.189 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T10:19:20.196 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-16T10:19:20.196 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-16T10:19:20.197 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T10:19:20.261 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f87b314fac0>, signals=[15]) 2024-09-16T10:19:20.261 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T10:19:20.270 INFO:teuthology.task.internal:Opening connections... 2024-09-16T10:19:20.270 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T10:19:20.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:19:20.354 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-09-16T10:19:20.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:19:20.429 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T10:19:20.438 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-09-16T10:19:20.457 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-09-16T10:19:20.457 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-09-16T10:19:20.515 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-09-16T10:19:20.515 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-09-16T10:19:20.515 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-09-16T10:19:20.515 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-09-16T10:19:20.516 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-09-16T10:19:20.516 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-09-16T10:19:20.516 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T10:19:20.516 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-09-16T10:19:20.516 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-09-16T10:19:20.516 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T10:19:20.516 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-09-16T10:19:20.516 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T10:19:20.516 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T10:19:20.516 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T10:19:20.517 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-09-16T10:19:20.545 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-09-16T10:19:20.564 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-09-16T10:19:20.565 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-09-16T10:19:20.623 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-09-16T10:19:20.623 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-09-16T10:19:20.623 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-09-16T10:19:20.623 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-09-16T10:19:20.624 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-09-16T10:19:20.624 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-09-16T10:19:20.624 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T10:19:20.624 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-09-16T10:19:20.624 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-09-16T10:19:20.624 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T10:19:20.624 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-09-16T10:19:20.624 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T10:19:20.624 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T10:19:20.624 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T10:19:20.625 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-09-16T10:19:20.645 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T10:19:20.658 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T10:19:20.670 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T10:19:20.670 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-09-16T10:19:20.673 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-09-16T10:19:20.689 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T10:19:20.700 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T10:19:20.701 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-09-16T10:19:20.731 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-09-16T10:19:20.818 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T10:19:20.875 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T10:19:20.894 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T10:19:20.894 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T10:19:20.894 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T10:19:20.895 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T10:19:20.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:19:20.912 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-16T10:19:20.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:19:20.914 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-16T10:19:20.970 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-503.el9.x86_64 2024-09-16T10:19:20.970 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.14.0-503.el9.x86_64 2024-09-16T10:19:20.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum install -y kernel 2024-09-16T10:19:20.971 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-503.el9.x86_64 2024-09-16T10:19:20.972 INFO:teuthology.task.kernel:Running kernel on smithi175: 5.14.0-503.el9.x86_64 2024-09-16T10:19:20.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum install -y kernel 2024-09-16T10:19:23.190 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-16T10:19:24.306 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-16T10:19:25.974 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-16T10:19:27.499 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 6.4 MB/s | 20 MB 00:03 2024-09-16T10:19:30.157 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-16T10:19:31.700 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 19 kB 00:00 2024-09-16T10:19:32.449 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.5 MB 00:01 2024-09-16T10:19:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-16T10:19:33.967 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-16T10:19:34.820 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-16T10:19:37.976 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-16T10:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T10:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T10:19:40.090 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-16T10:19:40.099 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-16T10:19:40.099 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-16T10:19:40.099 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-16T10:19:40.099 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-16T10:19:40.099 INFO:teuthology.orchestra.run.smithi175.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T10:19:40.099 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-16T10:19:40.099 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T10:19:40.099 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T10:19:40.100 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T10:19:40.100 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:19:40.100 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-16T10:19:40.100 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-16T10:19:40.100 INFO:teuthology.orchestra.run.smithi175.stdout:Install 4 Packages 2024-09-16T10:19:40.100 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:19:40.101 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 79 M 2024-09-16T10:19:40.101 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 126 M 2024-09-16T10:19:40.101 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-16T10:19:40.193 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-16T10:19:40.277 INFO:teuthology.orchestra.run.smithi175.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 290 kB/s | 23 kB 00:00 2024-09-16T10:19:40.636 INFO:teuthology.orchestra.run.smithi175.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-09-16T10:19:41.019 INFO:teuthology.orchestra.run.smithi175.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 42 MB/s | 35 MB 00:00 2024-09-16T10:19:41.395 INFO:teuthology.orchestra.run.smithi175.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 26 MB/s | 29 MB 00:01 2024-09-16T10:19:41.395 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-16T10:19:41.395 INFO:teuthology.orchestra.run.smithi175.stdout:Total 61 MB/s | 79 MB 00:01 2024-09-16T10:19:42.076 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-16T10:19:42.216 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T10:19:42.216 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T10:19:42.311 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-16T10:19:42.319 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-16T10:19:42.319 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-16T10:19:42.319 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-16T10:19:42.319 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-16T10:19:42.319 INFO:teuthology.orchestra.run.smithi164.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T10:19:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-16T10:19:42.320 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T10:19:42.320 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T10:19:42.320 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T10:19:42.320 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:19:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-16T10:19:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-16T10:19:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:Install 4 Packages 2024-09-16T10:19:42.320 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:19:42.321 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 79 M 2024-09-16T10:19:42.321 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 126 M 2024-09-16T10:19:42.321 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-16T10:19:42.417 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-16T10:19:42.418 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-16T10:19:42.686 INFO:teuthology.orchestra.run.smithi164.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 264 kB/s | 23 kB 00:00 2024-09-16T10:19:43.402 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-16T10:19:43.403 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-16T10:19:45.277 INFO:teuthology.orchestra.run.smithi164.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.8 MB/s | 16 MB 00:02 2024-09-16T10:19:45.856 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-16T10:19:46.561 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T10:19:46.644 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T10:19:46.778 INFO:teuthology.orchestra.run.smithi164.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 7.0 MB/s | 29 MB 00:04 2024-09-16T10:19:48.961 INFO:teuthology.orchestra.run.smithi164.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 5.4 MB/s | 35 MB 00:06 2024-09-16T10:19:48.961 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-16T10:19:48.961 INFO:teuthology.orchestra.run.smithi164.stdout:Total 12 MB/s | 79 MB 00:06 2024-09-16T10:19:49.455 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T10:19:49.589 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T10:19:49.644 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-16T10:19:49.995 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-16T10:19:49.996 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-16T10:19:50.968 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-16T10:19:50.968 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-16T10:19:53.433 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-16T10:19:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T10:19:54.178 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T10:19:54.400 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T10:19:54.447 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:19:57.006 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T10:19:57.131 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T10:19:59.292 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:20:01.971 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T10:20:02.271 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:20:07.153 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:20:37.754 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:20:37.804 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:20:38.149 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:20:38.149 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T10:20:38.149 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T10:20:38.149 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T10:20:39.022 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:20:39.022 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:20:39.022 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-16T10:20:39.022 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T10:20:39.022 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T10:20:39.022 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T10:20:39.022 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T10:20:39.022 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:20:39.022 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-16T10:20:39.415 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T10:20:39.495 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T10:20:39.496 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T10:20:39.496 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T10:20:39.496 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T10:20:39.496 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T10:20:39.496 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T10:20:39.496 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T10:20:39.496 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T10:20:39.554 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T10:20:39.555 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T10:20:39.555 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2024-09-16T10:20:39.590 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2024-09-16T10:20:39.590 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2024-09-16T10:20:39.590 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.06 2024-09-16T10:20:39.590 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 82.el9 2024-09-16T10:20:39.590 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:Group : Unspecified 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 8274670 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : builder@centos.org 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T10:20:39.591 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2024-09-16T10:20:39.592 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2024-09-16T10:20:39.592 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:20:39.592 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T10:20:39.592 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T10:20:39.592 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T10:20:39.592 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2024-09-16T10:20:39.592 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:20:39.592 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T10:20:39.593 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T10:20:39.593 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 5.14.0-505.el9.x86_64 2024-09-16T10:20:39.593 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T10:20:40.275 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-09-16T10:20:41.740 INFO:teuthology.orchestra.run.smithi175.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T10:20:41.761 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-09-16T10:20:41.767 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-09-16T10:20:41.785 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.3ZjuPFBNQe 2024-09-16T10:20:41.785 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3ZjuPFBNQe 2024-09-16T10:20:41.852 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.3ZjuPFBNQe 2024-09-16T10:20:41.992 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.3ZjuPFBNQe is 6KB 2024-09-16T10:20:42.040 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.3ZjuPFBNQe 2024-09-16T10:20:42.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T10:20:42.118 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T10:20:42.119 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T10:20:42.119 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T10:20:42.119 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T10:20:42.120 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T10:20:42.272 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-09-16T10:20:45.190 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:20:45.233 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:20:45.503 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:20:45.503 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T10:20:45.503 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T10:20:45.503 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T10:20:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:20:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:20:46.379 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-16T10:20:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T10:20:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T10:20:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T10:20:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T10:20:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:20:46.380 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-16T10:20:46.760 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T10:20:46.847 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T10:20:46.848 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T10:20:46.848 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T10:20:46.848 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T10:20:46.848 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T10:20:46.848 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T10:20:46.848 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T10:20:46.849 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T10:20:46.907 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T10:20:46.908 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T10:20:46.908 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2024-09-16T10:20:46.943 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2024-09-16T10:20:46.943 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2024-09-16T10:20:46.943 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.06 2024-09-16T10:20:46.943 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 82.el9 2024-09-16T10:20:46.943 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:Group : Unspecified 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 8274670 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : builder@centos.org 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : CentOS 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T10:20:46.944 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2024-09-16T10:20:46.945 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2024-09-16T10:20:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:20:46.945 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T10:20:46.945 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T10:20:46.945 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T10:20:46.945 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2024-09-16T10:20:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:20:46.945 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T10:20:46.946 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T10:20:46.946 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 5.14.0-505.el9.x86_64 2024-09-16T10:20:46.946 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T10:20:47.529 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-09-16T10:20:48.905 INFO:teuthology.orchestra.run.smithi164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T10:20:48.939 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-09-16T10:20:48.941 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-16T10:20:48.957 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.HgFOZyERXD 2024-09-16T10:20:48.957 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HgFOZyERXD 2024-09-16T10:20:49.026 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.HgFOZyERXD 2024-09-16T10:20:49.175 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.HgFOZyERXD is 6KB 2024-09-16T10:20:49.224 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.HgFOZyERXD 2024-09-16T10:20:49.239 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T10:20:49.306 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T10:20:49.307 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T10:20:49.307 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T10:20:49.307 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T10:20:49.308 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T10:20:49.463 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-09-16T10:20:49.542 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-09-16T10:21:12.297 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T10:21:12.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-16T10:21:12.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:21:19.469 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T10:21:19.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T10:21:19.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:21:54.109 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-16T10:22:03.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T10:22:03.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:22:12.311 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T10:22:21.317 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-16T10:22:21.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:22:21.660 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-16T10:22:22.018 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-16T10:22:22.018 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-16T10:22:22.018 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-16T10:22:22.033 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-505.el9.x86_64 2024-09-16T10:22:22.033 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T10:22:22.033 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T10:22:22.033 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T10:22:23.035 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T10:22:23.035 DEBUG:teuthology.orchestra.run.smithi175:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T10:22:23.116 INFO:teuthology.orchestra.run.smithi175.stdout:ttyS1 2024-09-16T10:22:23.137 DEBUG:teuthology.parallel:result is None 2024-09-16T10:23:03.121 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T10:23:15.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T10:23:15.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:23:15.388 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-16T10:23:15.729 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T10:23:15.729 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T10:23:15.729 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-16T10:23:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-505.el9.x86_64 2024-09-16T10:23:15.745 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T10:23:15.745 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T10:23:15.745 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T10:23:16.746 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T10:23:16.747 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T10:23:16.815 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-09-16T10:23:16.839 DEBUG:teuthology.parallel:result is None 2024-09-16T10:23:16.839 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T10:23:16.846 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T10:23:16.846 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T10:23:16.848 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T10:23:16.868 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T10:23:16.876 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T10:23:16.886 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T10:23:16.886 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T10:23:16.906 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T10:23:17.032 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T10:23:17.038 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T10:23:17.039 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T10:23:17.041 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T10:23:17.079 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T10:23:17.080 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T10:23:17.091 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T10:23:17.092 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T10:23:17.093 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T10:23:17.100 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T10:23:17.100 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T10:23:17.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T10:23:17.163 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T10:23:17.172 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T10:23:17.173 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T10:23:17.203 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T10:23:17.220 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T10:23:17.305 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T10:23:17.359 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T10:23:17.442 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T10:23:17.483 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:23:17.483 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T10:23:17.548 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T10:23:17.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T10:23:17.652 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T10:23:17.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T10:23:17.781 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:23:17.781 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T10:23:17.849 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-09-16T10:23:17.851 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-09-16T10:23:17.892 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T10:23:17.936 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T10:23:18.273 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T10:23:18.281 INFO:teuthology.task.internal:Starting timer... 2024-09-16T10:23:18.281 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T10:23:18.291 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T10:23:18.301 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-16T10:23:18.301 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-09-16T10:23:18.361 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-09-16T10:23:18.363 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-09-16T10:23:18.435 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-09-16T10:23:18.438 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T10:23:18.438 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-09-16T10:23:18.459 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-09-16T10:23:18.459 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-09-16T10:23:18.488 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-09-16T10:23:18.488 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi164.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-09-16T10:23:18.488 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T10:23:18.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:23:18.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T10:23:18.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:23:18.558 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T10:23:18.558 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-09-16T10:23:18.600 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-09-16T10:23:18.637 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T10:23:18.646 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T10:23:18.657 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T10:23:18.658 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi164.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T10:27:01.519 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi164.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-09-16T10:27:01.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T10:27:01.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:27:01.601 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-16T10:27:01.685 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T10:27:01.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-16T10:27:01.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:27:01.765 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-16T10:27:01.844 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-16T10:27:01.844 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T10:27:01.854 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T10:27:01.855 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T10:27:01.855 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T10:27:01.858 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T10:27:01.859 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T10:27:01.889 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T10:27:01.901 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T10:27:01.913 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T10:27:01.926 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-09-16T10:27:01.927 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T10:27:01.937 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-09-16T10:27:01.951 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T10:27:01.952 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-09-16T10:27:01.962 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-09-16T10:27:01.965 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T10:27:01.977 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T10:27:01.991 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T10:27:02.023 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-16T10:27:02.026 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T10:27:02.027 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-16T10:27:02.027 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:27:02.027 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:27:02.027 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:27:02.027 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:27:02.053 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-09-16T10:27:02.056 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T10:27:02.056 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-09-16T10:27:02.056 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:27:02.056 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:27:02.056 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:27:02.056 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:27:02.057 INFO:teuthology.run_tasks:Running task pexec... 2024-09-16T10:27:02.067 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-16T10:27:02.067 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T10:27:02.067 DEBUG:teuthology.orchestra.run.smithi164:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-16T10:27:02.068 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi175.front.sepia.ceph.com 2024-09-16T10:27:02.068 DEBUG:teuthology.orchestra.run.smithi175:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-16T10:27:02.418 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-16T10:27:02.432 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-16T10:27:02.432 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-16T10:27:02.432 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-16T10:27:02.432 INFO:teuthology.orchestra.run.smithi164.stdout:Removing: 2024-09-16T10:27:02.432 INFO:teuthology.orchestra.run.smithi164.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-16T10:27:02.432 INFO:teuthology.orchestra.run.smithi164.stdout:Removing unused dependencies: 2024-09-16T10:27:02.432 INFO:teuthology.orchestra.run.smithi164.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-16T10:27:02.433 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:27:02.433 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-16T10:27:02.433 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-16T10:27:02.433 INFO:teuthology.orchestra.run.smithi164.stdout:Remove 2 Packages 2024-09-16T10:27:02.433 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:27:02.433 INFO:teuthology.orchestra.run.smithi164.stdout:Freed space: 5.5 M 2024-09-16T10:27:02.433 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-16T10:27:02.436 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-16T10:27:02.436 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-16T10:27:02.448 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-16T10:27:02.456 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-16T10:27:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-16T10:27:02.456 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-16T10:27:02.456 INFO:teuthology.orchestra.run.smithi175.stdout:Removing: 2024-09-16T10:27:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-16T10:27:02.456 INFO:teuthology.orchestra.run.smithi175.stdout:Removing unused dependencies: 2024-09-16T10:27:02.456 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-16T10:27:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:27:02.457 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-16T10:27:02.457 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-16T10:27:02.457 INFO:teuthology.orchestra.run.smithi175.stdout:Remove 2 Packages 2024-09-16T10:27:02.457 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:27:02.457 INFO:teuthology.orchestra.run.smithi175.stdout:Freed space: 5.5 M 2024-09-16T10:27:02.457 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-16T10:27:02.460 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-16T10:27:02.461 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-16T10:27:02.505 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-16T10:27:02.505 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-16T10:27:02.522 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-16T10:27:02.523 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-16T10:27:02.644 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-16T10:27:02.657 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-16T10:27:02.725 INFO:teuthology.orchestra.run.smithi164.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-16T10:27:02.744 INFO:teuthology.orchestra.run.smithi175.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-16T10:27:02.773 INFO:teuthology.orchestra.run.smithi164.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T10:27:02.813 INFO:teuthology.orchestra.run.smithi175.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T10:27:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T10:27:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-16T10:27:03.333 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T10:27:03.333 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-16T10:27:03.533 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-16T10:27:03.533 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:27:03.533 INFO:teuthology.orchestra.run.smithi164.stdout:Removed: 2024-09-16T10:27:03.533 INFO:teuthology.orchestra.run.smithi164.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T10:27:03.533 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:27:03.534 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-16T10:27:03.588 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-16T10:27:03.588 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:27:03.589 INFO:teuthology.orchestra.run.smithi175.stdout:Removed: 2024-09-16T10:27:03.589 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T10:27:03.589 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:27:03.589 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-16T10:27:04.319 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:47 ago on Mon 16 Sep 2024 10:26:17 AM UTC. 2024-09-16T10:27:04.419 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:47 ago on Mon 16 Sep 2024 10:26:17 AM UTC. 2024-09-16T10:27:04.456 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-16T10:27:04.456 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:27:04.457 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-16T10:27:04.458 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-16T10:27:04.458 INFO:teuthology.orchestra.run.smithi164.stdout:Install 7 Packages 2024-09-16T10:27:04.458 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:27:04.475 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 5.2 M 2024-09-16T10:27:04.475 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 19 M 2024-09-16T10:27:04.475 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-16T10:27:04.556 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-16T10:27:04.557 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-16T10:27:04.557 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-16T10:27:04.557 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-16T10:27:04.558 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-16T10:27:04.558 INFO:teuthology.orchestra.run.smithi175.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-16T10:27:04.558 INFO:teuthology.orchestra.run.smithi175.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-16T10:27:04.558 INFO:teuthology.orchestra.run.smithi175.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-16T10:27:04.558 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-16T10:27:04.558 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-16T10:27:04.558 INFO:teuthology.orchestra.run.smithi175.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-16T10:27:04.559 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-16T10:27:04.559 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-16T10:27:04.559 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:27:04.559 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-16T10:27:04.559 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-16T10:27:04.559 INFO:teuthology.orchestra.run.smithi175.stdout:Install 7 Packages 2024-09-16T10:27:04.560 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:27:04.566 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 5.2 M 2024-09-16T10:27:04.566 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 19 M 2024-09-16T10:27:04.566 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-16T10:27:04.654 INFO:teuthology.orchestra.run.smithi164.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 427 kB/s | 44 kB 00:00 2024-09-16T10:27:04.679 INFO:teuthology.orchestra.run.smithi164.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 818 kB/s | 106 kB 00:00 2024-09-16T10:27:04.738 INFO:teuthology.orchestra.run.smithi164.stdout:(3/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.7 MB/s | 906 kB 00:00 2024-09-16T10:27:04.763 INFO:teuthology.orchestra.run.smithi164.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 658 kB/s | 72 kB 00:00 2024-09-16T10:27:04.830 INFO:teuthology.orchestra.run.smithi164.stdout:(5/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 8.9 MB/s | 837 kB 00:00 2024-09-16T10:27:04.914 INFO:teuthology.orchestra.run.smithi164.stdout:(6/7): python3-kmod-0.9-32.el9.x86_64.rpm 359 kB/s | 84 kB 00:00 2024-09-16T10:27:05.423 INFO:teuthology.orchestra.run.smithi164.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 4.8 MB/s | 3.2 MB 00:00 2024-09-16T10:27:05.423 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-16T10:27:05.423 INFO:teuthology.orchestra.run.smithi164.stdout:Total 5.4 MB/s | 5.2 MB 00:00 2024-09-16T10:27:05.529 INFO:teuthology.orchestra.run.smithi175.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 75 kB/s | 44 kB 00:00 2024-09-16T10:27:05.534 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-16T10:27:05.549 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-16T10:27:05.550 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-16T10:27:05.654 INFO:teuthology.orchestra.run.smithi175.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 147 kB/s | 106 kB 00:00 2024-09-16T10:27:05.829 INFO:teuthology.orchestra.run.smithi175.stdout:(3/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 1.0 MB/s | 906 kB 00:00 2024-09-16T10:27:05.870 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-16T10:27:05.870 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-16T10:27:06.163 INFO:teuthology.orchestra.run.smithi175.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 113 kB/s | 72 kB 00:00 2024-09-16T10:27:06.338 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-16T10:27:06.396 INFO:teuthology.orchestra.run.smithi175.stdout:(5/7): runc-1.1.13-4.el9.x86_64.rpm 14 MB/s | 3.2 MB 00:00 2024-09-16T10:27:06.422 INFO:teuthology.orchestra.run.smithi175.stdout:(6/7): python3-kmod-0.9-32.el9.x86_64.rpm 110 kB/s | 84 kB 00:00 2024-09-16T10:27:06.425 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-16T10:27:06.460 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-16T10:27:06.488 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-16T10:27:06.697 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-16T10:27:06.714 INFO:teuthology.orchestra.run.smithi175.stdout:(7/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 949 kB/s | 837 kB 00:00 2024-09-16T10:27:06.714 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-16T10:27:06.714 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.4 MB/s | 5.2 MB 00:02 2024-09-16T10:27:06.728 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T10:27:06.823 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-16T10:27:06.838 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-16T10:27:06.839 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-16T10:27:07.154 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-16T10:27:07.155 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-16T10:27:07.163 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T10:27:07.197 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T10:27:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T10:27:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T10:27:07.620 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-16T10:27:07.700 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-16T10:27:07.734 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-16T10:27:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-16T10:27:07.970 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-16T10:27:08.002 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T10:27:08.340 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T10:27:08.341 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-16T10:27:08.341 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-16T10:27:08.341 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-16T10:27:08.341 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-16T10:27:08.341 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-16T10:27:08.341 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-16T10:27:08.449 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T10:27:08.471 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T10:27:08.588 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T10:27:08.649 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T10:27:08.701 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T10:27:08.701 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:27:08.701 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-16T10:27:08.701 INFO:teuthology.orchestra.run.smithi164.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T10:27:08.701 INFO:teuthology.orchestra.run.smithi164.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-16T10:27:08.701 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-16T10:27:08.701 INFO:teuthology.orchestra.run.smithi164.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-16T10:27:08.701 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:27:08.701 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-16T10:27:09.044 DEBUG:teuthology.parallel:result is None 2024-09-16T10:27:09.616 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T10:27:09.616 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-16T10:27:09.616 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-16T10:27:09.616 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-16T10:27:09.616 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-16T10:27:09.616 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-16T10:27:09.616 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-16T10:27:10.114 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T10:27:10.114 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:27:10.114 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-16T10:27:10.115 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T10:27:10.115 INFO:teuthology.orchestra.run.smithi175.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-16T10:27:10.115 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-16T10:27:10.115 INFO:teuthology.orchestra.run.smithi175.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-16T10:27:10.115 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:27:10.115 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-16T10:27:10.504 DEBUG:teuthology.parallel:result is None 2024-09-16T10:27:10.504 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-09-16T10:27:10.523 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-09-16T10:27:10.523 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:27:10.523 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T10:27:10.543 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T10:27:10.543 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-09-16T10:27:10.602 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T10:27:10.603 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T10:27:10.603 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-16T10:27:10.603 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T10:27:10.603 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T10:27:10.603 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 10:27:07.611585666 +0000 2024-09-16T10:27:10.603 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 10:27:07.271557532 +0000 2024-09-16T10:27:10.603 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 10:27:07.271557532 +0000 2024-09-16T10:27:10.603 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 10:27:07.271557532 +0000 2024-09-16T10:27:10.603 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T10:27:10.671 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T10:27:10.671 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T10:27:10.671 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000223603 s, 2.3 MB/s 2024-09-16T10:27:10.673 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T10:27:10.731 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-09-16T10:27:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T10:27:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T10:27:10.787 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 961 Links: 1 2024-09-16T10:27:10.787 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T10:27:10.787 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T10:27:10.787 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 10:27:07.611585666 +0000 2024-09-16T10:27:10.787 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 10:27:07.271557532 +0000 2024-09-16T10:27:10.788 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 10:27:07.271557532 +0000 2024-09-16T10:27:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 10:27:07.271557532 +0000 2024-09-16T10:27:10.788 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T10:27:10.851 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T10:27:10.851 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T10:27:10.852 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000224837 s, 2.3 MB/s 2024-09-16T10:27:10.853 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T10:27:10.909 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-09-16T10:27:10.965 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T10:27:10.965 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T10:27:10.965 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-16T10:27:10.965 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T10:27:10.966 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T10:27:10.966 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 10:27:07.612585748 +0000 2024-09-16T10:27:10.966 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 10:27:07.272557615 +0000 2024-09-16T10:27:10.966 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 10:27:07.272557615 +0000 2024-09-16T10:27:10.966 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 10:27:07.272557615 +0000 2024-09-16T10:27:10.967 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T10:27:11.029 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T10:27:11.029 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T10:27:11.029 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000178034 s, 2.9 MB/s 2024-09-16T10:27:11.030 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T10:27:11.086 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-09-16T10:27:11.142 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T10:27:11.142 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T10:27:11.142 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-16T10:27:11.142 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T10:27:11.142 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T10:27:11.142 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 10:27:07.612585748 +0000 2024-09-16T10:27:11.142 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 10:27:07.274557780 +0000 2024-09-16T10:27:11.142 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 10:27:07.274557780 +0000 2024-09-16T10:27:11.142 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 10:27:07.274557780 +0000 2024-09-16T10:27:11.143 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T10:27:11.205 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T10:27:11.205 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T10:27:11.205 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000195869 s, 2.6 MB/s 2024-09-16T10:27:11.206 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T10:27:11.262 DEBUG:teuthology.orchestra.run.smithi164:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-16T10:27:11.399 INFO:teuthology.orchestra.run.smithi164.stdout:loop 2024-09-16T10:27:11.401 INFO:tasks.nvme_loop:Connecting nvme_loop smithi164:/dev/vg_nvme/lv_1... 2024-09-16T10:27:11.402 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-16T10:27:11.434 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-09-16T10:27:11.464 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/vg_nvme/lv_11 2024-09-16T10:27:11.509 INFO:teuthology.orchestra.run.smithi164.stdout:connecting to device: nvme1 2024-09-16T10:27:11.511 INFO:tasks.nvme_loop:Connecting nvme_loop smithi164:/dev/vg_nvme/lv_2... 2024-09-16T10:27:11.511 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-16T10:27:11.588 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-09-16T10:27:11.619 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/vg_nvme/lv_21 2024-09-16T10:27:11.650 INFO:teuthology.orchestra.run.smithi164.stdout:connecting to device: nvme2 2024-09-16T10:27:11.651 INFO:tasks.nvme_loop:Connecting nvme_loop smithi164:/dev/vg_nvme/lv_3... 2024-09-16T10:27:11.652 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-16T10:27:11.726 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-09-16T10:27:11.756 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/vg_nvme/lv_31 2024-09-16T10:27:11.787 INFO:teuthology.orchestra.run.smithi164.stdout:connecting to device: nvme3 2024-09-16T10:27:11.789 INFO:tasks.nvme_loop:Connecting nvme_loop smithi164:/dev/vg_nvme/lv_4... 2024-09-16T10:27:11.789 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-16T10:27:11.863 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-09-16T10:27:11.893 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/vg_nvme/lv_41 2024-09-16T10:27:11.924 INFO:teuthology.orchestra.run.smithi164.stdout:connecting to device: nvme4 2024-09-16T10:27:11.928 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:27:11.928 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T10:27:11.983 DEBUG:teuthology.orchestra.run.smithi164:> sudo nvme list -o json 2024-09-16T10:27:12.058 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T10:27:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: "Devices":[ 2024-09-16T10:27:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T10:27:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: "NameSpace":1, 2024-09-16T10:27:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-16T10:27:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: "GenericPath":"/dev/ng4n1", 2024-09-16T10:27:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: "Firmware":"5.14.0-5", 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "ModelNumber":"Linux", 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "SerialNumber":"8aa0d2e26cbdb8078d05", 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "UsedBytes":95995035648, 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "MaximumLBA":187490304, 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "PhysicalSize":95995035648, 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "SectorSize":512 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "NameSpace":1, 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "GenericPath":"/dev/ng3n1", 2024-09-16T10:27:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "Firmware":"5.14.0-5", 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "ModelNumber":"Linux", 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "SerialNumber":"f88aa3bd38a9077199db", 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "UsedBytes":95995035648, 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "MaximumLBA":187490304, 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "PhysicalSize":95995035648, 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "SectorSize":512 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "NameSpace":1, 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "GenericPath":"/dev/ng2n1", 2024-09-16T10:27:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "Firmware":"5.14.0-5", 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "ModelNumber":"Linux", 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "SerialNumber":"e1380edeeeaf39d69a5f", 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "UsedBytes":95995035648, 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "MaximumLBA":187490304, 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "PhysicalSize":95995035648, 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "SectorSize":512 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "NameSpace":1, 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "GenericPath":"/dev/ng1n1", 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "Firmware":"5.14.0-5", 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "ModelNumber":"Linux", 2024-09-16T10:27:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "SerialNumber":"3cbbed3bfe7af15735a8", 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "UsedBytes":95995035648, 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "MaximumLBA":187490304, 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "PhysicalSize":95995035648, 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "SectorSize":512 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "NameSpace":1, 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "GenericPath":"/dev/ng0n1", 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "Firmware":"8DV101H0", 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "SerialNumber":"CVFT623300H0400BGN", 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "UsedBytes":400088457216, 2024-09-16T10:27:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "MaximumLBA":781422768, 2024-09-16T10:27:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: "PhysicalSize":400088457216, 2024-09-16T10:27:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: "SectorSize":512 2024-09-16T10:27:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T10:27:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T10:27:12.064 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T10:27:12.065 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T10:27:12.065 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:27:12.065 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/scratch_devs 2024-09-16T10:27:12.131 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:27:12.131 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T10:27:12.151 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T10:27:12.151 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-09-16T10:27:12.210 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T10:27:12.210 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T10:27:12.210 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-16T10:27:12.210 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T10:27:12.210 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T10:27:12.211 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-16 10:27:08.861362412 +0000 2024-09-16T10:27:12.211 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.211 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.211 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.211 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T10:27:12.277 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-16T10:27:12.277 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-16T10:27:12.278 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000192683 s, 2.7 MB/s 2024-09-16T10:27:12.279 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T10:27:12.338 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-09-16T10:27:12.394 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T10:27:12.395 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T10:27:12.395 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 980 Links: 1 2024-09-16T10:27:12.395 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T10:27:12.395 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T10:27:12.395 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-16 10:27:08.861362412 +0000 2024-09-16T10:27:12.395 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.395 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.395 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T10:27:12.459 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-16T10:27:12.459 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-16T10:27:12.459 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000215463 s, 2.4 MB/s 2024-09-16T10:27:12.460 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T10:27:12.517 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-09-16T10:27:12.575 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T10:27:12.575 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T10:27:12.575 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-16T10:27:12.575 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T10:27:12.576 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T10:27:12.576 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-16 10:27:08.861362412 +0000 2024-09-16T10:27:12.576 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.576 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.576 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.577 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T10:27:12.639 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-16T10:27:12.639 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-16T10:27:12.639 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000237148 s, 2.2 MB/s 2024-09-16T10:27:12.640 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T10:27:12.698 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-09-16T10:27:12.753 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T10:27:12.753 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T10:27:12.754 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-16T10:27:12.754 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T10:27:12.754 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T10:27:12.754 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-16 10:27:08.862362397 +0000 2024-09-16T10:27:12.754 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.754 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.754 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-16 10:27:08.556366916 +0000 2024-09-16T10:27:12.754 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T10:27:12.817 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-16T10:27:12.817 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-16T10:27:12.817 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000246469 s, 2.1 MB/s 2024-09-16T10:27:12.818 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T10:27:12.875 DEBUG:teuthology.orchestra.run.smithi175:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-16T10:27:13.028 INFO:teuthology.orchestra.run.smithi175.stdout:loop 2024-09-16T10:27:13.029 INFO:tasks.nvme_loop:Connecting nvme_loop smithi175:/dev/vg_nvme/lv_1... 2024-09-16T10:27:13.029 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-16T10:27:13.061 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-16T10:27:13.091 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/vg_nvme/lv_11 2024-09-16T10:27:13.134 INFO:teuthology.orchestra.run.smithi175.stdout:connecting to device: nvme1 2024-09-16T10:27:13.135 INFO:tasks.nvme_loop:Connecting nvme_loop smithi175:/dev/vg_nvme/lv_2... 2024-09-16T10:27:13.136 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-16T10:27:13.214 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-16T10:27:13.244 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/vg_nvme/lv_21 2024-09-16T10:27:13.276 INFO:teuthology.orchestra.run.smithi175.stdout:connecting to device: nvme2 2024-09-16T10:27:13.278 INFO:tasks.nvme_loop:Connecting nvme_loop smithi175:/dev/vg_nvme/lv_3... 2024-09-16T10:27:13.278 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-16T10:27:13.353 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-16T10:27:13.384 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/vg_nvme/lv_31 2024-09-16T10:27:13.419 INFO:teuthology.orchestra.run.smithi175.stdout:connecting to device: nvme3 2024-09-16T10:27:13.422 INFO:tasks.nvme_loop:Connecting nvme_loop smithi175:/dev/vg_nvme/lv_4... 2024-09-16T10:27:13.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-16T10:27:13.495 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-16T10:27:13.525 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/vg_nvme/lv_41 2024-09-16T10:27:13.556 INFO:teuthology.orchestra.run.smithi175.stdout:connecting to device: nvme4 2024-09-16T10:27:13.562 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:27:13.562 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T10:27:13.619 DEBUG:teuthology.orchestra.run.smithi175:> sudo nvme list -o json 2024-09-16T10:27:13.687 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-16T10:27:13.687 INFO:teuthology.orchestra.run.smithi175.stdout: "Devices":[ 2024-09-16T10:27:13.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-16T10:27:13.687 INFO:teuthology.orchestra.run.smithi175.stdout: "NameSpace":1, 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "GenericPath":"/dev/ng4n1", 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "Firmware":"5.14.0-5", 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "ModelNumber":"Linux", 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "SerialNumber":"67281d28ca84bcde9cfa", 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "UsedBytes":95995035648, 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "MaximumLBA":187490304, 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "PhysicalSize":95995035648, 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "SectorSize":512 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-16T10:27:13.688 INFO:teuthology.orchestra.run.smithi175.stdout: "NameSpace":1, 2024-09-16T10:27:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-16T10:27:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "GenericPath":"/dev/ng3n1", 2024-09-16T10:27:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "Firmware":"5.14.0-5", 2024-09-16T10:27:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "ModelNumber":"Linux", 2024-09-16T10:27:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "SerialNumber":"6e5cbfe78f644349609f", 2024-09-16T10:27:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "UsedBytes":95995035648, 2024-09-16T10:27:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "MaximumLBA":187490304, 2024-09-16T10:27:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "PhysicalSize":95995035648, 2024-09-16T10:27:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: "SectorSize":512 2024-09-16T10:27:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-16T10:27:13.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "NameSpace":1, 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "GenericPath":"/dev/ng2n1", 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "Firmware":"5.14.0-5", 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "ModelNumber":"Linux", 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "SerialNumber":"e7977759e4910d9f0efe", 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "UsedBytes":95995035648, 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "MaximumLBA":187490304, 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "PhysicalSize":95995035648, 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: "SectorSize":512 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-16T10:27:13.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-16T10:27:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "NameSpace":1, 2024-09-16T10:27:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-16T10:27:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "GenericPath":"/dev/ng1n1", 2024-09-16T10:27:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "Firmware":"5.14.0-5", 2024-09-16T10:27:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "ModelNumber":"Linux", 2024-09-16T10:27:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "SerialNumber":"63e52e71259bdbfe9151", 2024-09-16T10:27:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "UsedBytes":95995035648, 2024-09-16T10:27:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "MaximumLBA":187490304, 2024-09-16T10:27:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "PhysicalSize":95995035648, 2024-09-16T10:27:13.691 INFO:teuthology.orchestra.run.smithi175.stdout: "SectorSize":512 2024-09-16T10:27:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-16T10:27:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-16T10:27:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "NameSpace":1, 2024-09-16T10:27:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-16T10:27:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "GenericPath":"/dev/ng0n1", 2024-09-16T10:27:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "Firmware":"8DV101H0", 2024-09-16T10:27:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-16T10:27:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "SerialNumber":"CVFT623300M0400BGN", 2024-09-16T10:27:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "UsedBytes":400088457216, 2024-09-16T10:27:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "MaximumLBA":781422768, 2024-09-16T10:27:13.692 INFO:teuthology.orchestra.run.smithi175.stdout: "PhysicalSize":400088457216, 2024-09-16T10:27:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: "SectorSize":512 2024-09-16T10:27:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-16T10:27:13.693 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-16T10:27:13.693 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-16T10:27:13.694 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T10:27:13.694 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:27:13.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/scratch_devs 2024-09-16T10:27:13.758 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T10:27:13.855 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 3}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'MON_DOWN', 'mons down', 'mon down', 'out of quorum', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'CEPHADM_AGENT_DOWN'], 'log-only-match': ['CEPHADM_'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'} 2024-09-16T10:27:13.856 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T10:27:13.856 INFO:tasks.cephadm:Cluster fsid is 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:27:13.856 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T10:27:13.857 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.164', 'mon.c': '[v2:172.21.15.164:3301,v1:172.21.15.164:6790]', 'mon.b': '172.21.15.175'} 2024-09-16T10:27:13.857 INFO:tasks.cephadm:First mon is mon.a on smithi164 2024-09-16T10:27:13.857 INFO:tasks.cephadm:First mgr is y 2024-09-16T10:27:13.857 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T10:27:13.857 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname $(hostname -s) 2024-09-16T10:27:13.888 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2024-09-16T10:27:13.913 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-16T10:27:13.913 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T10:27:14.059 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56072/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-845-ge4e227b3', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-845.ge4e227b3'}, 'url': 'https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-13 17:59:03.516496', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-16T10:27:14.206 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 e4e227b37858f5109138494e01e5ba1ba7b8ffe8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T10:27:14.207 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/x86_64/flavors/default/cephadm 2024-09-16T10:27:14.208 DEBUG:teuthology.orchestra.run.smithi164:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T10:27:14.566 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 16 10:27 /home/ubuntu/cephtest/cephadm 2024-09-16T10:27:14.566 DEBUG:teuthology.orchestra.run.smithi175:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T10:27:14.902 INFO:teuthology.orchestra.run.smithi175.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 16 10:27 /home/ubuntu/cephtest/cephadm 2024-09-16T10:27:14.902 DEBUG:teuthology.orchestra.run.smithi164:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T10:27:14.922 DEBUG:teuthology.orchestra.run.smithi175:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T10:27:14.951 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 on all hosts... 2024-09-16T10:27:14.951 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 pull 2024-09-16T10:27:14.965 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 pull 2024-09-16T10:27:16.394 INFO:teuthology.orchestra.run.smithi175.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T10:27:16.467 INFO:teuthology.orchestra.run.smithi164.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T10:27:43.362 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T10:27:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable)", 2024-09-16T10:27:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "image_id": "67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1", 2024-09-16T10:27:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "repo_digests": [ 2024-09-16T10:27:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322" 2024-09-16T10:27:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T10:27:43.363 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T10:27:44.756 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-16T10:27:44.756 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable)", 2024-09-16T10:27:44.756 INFO:teuthology.orchestra.run.smithi175.stdout: "image_id": "67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1", 2024-09-16T10:27:44.756 INFO:teuthology.orchestra.run.smithi175.stdout: "repo_digests": [ 2024-09-16T10:27:44.757 INFO:teuthology.orchestra.run.smithi175.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322" 2024-09-16T10:27:44.757 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-16T10:27:44.757 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-16T10:27:44.775 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph 2024-09-16T10:27:44.809 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2024-09-16T10:27:44.836 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /etc/ceph 2024-09-16T10:27:44.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2024-09-16T10:27:44.902 INFO:tasks.cephadm:Writing seed config... 2024-09-16T10:27:44.903 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T10:27:44.903 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T10:27:44.904 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-09-16T10:27:44.904 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-16T10:27:44.904 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T10:27:44.904 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T10:27:44.904 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T10:27:44.904 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T10:27:44.904 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T10:27:44.904 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T10:27:44.905 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:27:44.905 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T10:27:44.935 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3d5087c4-7416-11ef-bceb-c7b262605968 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-16T10:27:44.936 DEBUG:teuthology.orchestra.run.smithi164:mon.a> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T10:27:44.978 DEBUG:teuthology.orchestra.run.smithi164:mgr.y> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T10:27:45.021 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T10:27:45.021 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 -v bootstrap --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.164 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T10:27:45.175 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-16T10:27:45.176 INFO:teuthology.orchestra.run.smithi164.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8', '-v', 'bootstrap', '--fsid', '3d5087c4-7416-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.164', '--skip-admin-label'] 2024-09-16T10:27:45.199 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stdout 5.2.2 2024-09-16T10:27:45.199 INFO:teuthology.orchestra.run.smithi164.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T10:27:45.200 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying podman|docker is present... 2024-09-16T10:27:45.222 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stdout 5.2.2 2024-09-16T10:27:45.222 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying lvm2 is present... 2024-09-16T10:27:45.222 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying time synchronization is in place... 2024-09-16T10:27:45.230 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T10:27:45.230 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T10:27:45.236 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T10:27:45.236 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-16T10:27:45.242 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout enabled 2024-09-16T10:27:45.249 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout active 2024-09-16T10:27:45.249 INFO:teuthology.orchestra.run.smithi164.stdout:Unit chronyd.service is enabled and running 2024-09-16T10:27:45.249 INFO:teuthology.orchestra.run.smithi164.stdout:Repeating the final host check... 2024-09-16T10:27:45.270 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stdout 5.2.2 2024-09-16T10:27:45.270 INFO:teuthology.orchestra.run.smithi164.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T10:27:45.270 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl is present 2024-09-16T10:27:45.270 INFO:teuthology.orchestra.run.smithi164.stdout:lvcreate is present 2024-09-16T10:27:45.276 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T10:27:45.277 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T10:27:45.283 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T10:27:45.283 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-16T10:27:45.289 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout enabled 2024-09-16T10:27:45.296 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout active 2024-09-16T10:27:45.296 INFO:teuthology.orchestra.run.smithi164.stdout:Unit chronyd.service is enabled and running 2024-09-16T10:27:45.296 INFO:teuthology.orchestra.run.smithi164.stdout:Host looks OK 2024-09-16T10:27:45.296 INFO:teuthology.orchestra.run.smithi164.stdout:Cluster fsid: 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:27:45.296 INFO:teuthology.orchestra.run.smithi164.stdout:Acquiring lock 140496073519264 on /run/cephadm/3d5087c4-7416-11ef-bceb-c7b262605968.lock 2024-09-16T10:27:45.296 INFO:teuthology.orchestra.run.smithi164.stdout:Lock 140496073519264 acquired on /run/cephadm/3d5087c4-7416-11ef-bceb-c7b262605968.lock 2024-09-16T10:27:45.297 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying IP 172.21.15.164 port 3300 ... 2024-09-16T10:27:45.297 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying IP 172.21.15.164 port 6789 ... 2024-09-16T10:27:45.297 INFO:teuthology.orchestra.run.smithi164.stdout:Base mon IP(s) is [172.21.15.164:3300, 172.21.15.164:6789], mon addrv is [v2:172.21.15.164:3300,v1:172.21.15.164:6789] 2024-09-16T10:27:45.300 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.164 metric 100 2024-09-16T10:27:45.300 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.164 metric 100 2024-09-16T10:27:45.303 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T10:27:45.303 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-16T10:27:45.303 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-16T10:27:45.305 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T10:27:45.305 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T10:27:45.305 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T10:27:45.305 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-16T10:27:45.305 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:73f3/64 scope link noprefixroute 2024-09-16T10:27:45.306 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T10:27:45.306 INFO:teuthology.orchestra.run.smithi164.stdout:Mon IP `172.21.15.164` is in CIDR network `172.21.0.0/20` 2024-09-16T10:27:45.307 INFO:teuthology.orchestra.run.smithi164.stdout:Mon IP `172.21.15.164` is in CIDR network `172.21.0.0/20` 2024-09-16T10:27:45.307 INFO:teuthology.orchestra.run.smithi164.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T10:27:45.307 INFO:teuthology.orchestra.run.smithi164.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T10:27:45.307 INFO:teuthology.orchestra.run.smithi164.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T10:27:45.798 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stdout 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 2024-09-16T10:27:45.798 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8... 2024-09-16T10:27:45.798 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T10:27:45.798 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Copying blob sha256:71878612010b3dfa9e3e456d432a49e20ac26634831a8a6522f4d782181c5840 2024-09-16T10:27:45.798 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T10:27:45.798 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Copying config sha256:67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 2024-09-16T10:27:45.798 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T10:27:46.703 INFO:teuthology.orchestra.run.smithi164.stdout:ceph: stdout ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable) 2024-09-16T10:27:46.703 INFO:teuthology.orchestra.run.smithi164.stdout:Ceph version: ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable) 2024-09-16T10:27:46.703 INFO:teuthology.orchestra.run.smithi164.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T10:27:47.583 INFO:teuthology.orchestra.run.smithi164.stdout:stat: stdout 167 167 2024-09-16T10:27:47.583 INFO:teuthology.orchestra.run.smithi164.stdout:Creating initial keys... 2024-09-16T10:27:48.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph-authtool: stdout AQAjCOhmRnIhNxAAn5KbjDFnlq9QdFtN40rlnw== 2024-09-16T10:27:49.529 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph-authtool: stdout AQAkCOhmZVECORAAF/TNQaVkQtiIJQbx7QYomA== 2024-09-16T10:27:50.534 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph-authtool: stdout AQAlCOhmpRlhNBAAhgsiAG80l68ZgrY3B9ZeDQ== 2024-09-16T10:27:50.535 INFO:teuthology.orchestra.run.smithi164.stdout:Creating initial monmap... 2024-09-16T10:27:51.490 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T10:27:51.490 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-16T10:27:51.490 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:27:51.490 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T10:27:51.491 INFO:teuthology.orchestra.run.smithi164.stdout:monmaptool for a [v2:172.21.15.164:3300,v1:172.21.15.164:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T10:27:51.491 INFO:teuthology.orchestra.run.smithi164.stdout:setting min_mon_release = pacific 2024-09-16T10:27:51.491 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: set fsid to 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:27:51.491 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T10:27:51.491 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:27:51.491 INFO:teuthology.orchestra.run.smithi164.stdout:Creating mon... 2024-09-16T10:27:52.404 INFO:teuthology.orchestra.run.smithi164.stdout:create mon.a on 2024-09-16T10:27:52.762 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T10:27:52.946 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3d5087c4-7416-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-3d5087c4-7416-11ef-bceb-c7b262605968.target. 2024-09-16T10:27:52.946 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3d5087c4-7416-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-3d5087c4-7416-11ef-bceb-c7b262605968.target. 2024-09-16T10:27:53.165 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.a 2024-09-16T10:27:53.165 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T10:27:53.337 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3d5087c4-7416-11ef-bceb-c7b262605968.target.wants/ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-3d5087c4-7416-11ef-bceb-c7b262605968@.service. 2024-09-16T10:27:53.822 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:53 smithi164 ceph-mon[25359]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T10:27:53.836 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T10:27:53.836 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout disabled 2024-09-16T10:27:53.843 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T10:27:53.843 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-16T10:27:53.843 INFO:teuthology.orchestra.run.smithi164.stdout:firewalld.service is not enabled 2024-09-16T10:27:53.843 INFO:teuthology.orchestra.run.smithi164.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T10:27:53.843 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mon to start... 2024-09-16T10:27:53.843 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mon... 2024-09-16T10:27:54.091 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:53 smithi164 bash[25321]: ed97711e9992994f68fc81831d1d4c485a4ccb63d7dffd4bd4444e090d7228bc 2024-09-16T10:27:54.091 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:53 smithi164 systemd[1]: Started Ceph mon.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:27:54.912 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:54 smithi164 ceph-mon[25359]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T10:27:54.912 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:54 smithi164 ceph-mon[25359]: monmap e1: 1 mons at {a=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T10:27:54.912 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:54 smithi164 ceph-mon[25359]: fsmap 2024-09-16T10:27:54.912 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:54 smithi164 ceph-mon[25359]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T10:27:54.913 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:54 smithi164 ceph-mon[25359]: mgrmap e1: no daemons active 2024-09-16T10:27:54.913 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:54 smithi164 ceph-mon[25359]: from='client.? 172.21.15.164:0/3782231484' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T10:27:54.915 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout id: 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout services: 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.572772s) 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout data: 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T10:27:54.916 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T10:27:54.917 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T10:27:54.917 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:27:54.917 INFO:teuthology.orchestra.run.smithi164.stdout:mon is available 2024-09-16T10:27:54.917 INFO:teuthology.orchestra.run.smithi164.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T10:27:55.972 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:27:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T10:27:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout fsid = 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:27:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.164:3300,v1:172.21.15.164:6789] 2024-09-16T10:27:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T10:27:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T10:27:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T10:27:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T10:27:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:27:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T10:27:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-09-16T10:27:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T10:27:55.974 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:27:55.974 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T10:27:55.974 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T10:27:55.974 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T10:27:55.974 INFO:teuthology.orchestra.run.smithi164.stdout:Generating new minimal ceph.conf... 2024-09-16T10:27:56.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:55 smithi164 ceph-mon[25359]: from='client.? 172.21.15.164:0/1395427663' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T10:27:56.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:55 smithi164 ceph-mon[25359]: from='client.? 172.21.15.164:0/1395427663' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T10:27:57.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:56 smithi164 ceph-mon[25359]: from='client.? 172.21.15.164:0/2776939713' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:27:57.035 INFO:teuthology.orchestra.run.smithi164.stdout:Restarting the monitor... 2024-09-16T10:27:57.305 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:57 smithi164 systemd[1]: Stopping Ceph mon.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:27:57.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:57 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a[25336]: 2024-09-16T10:27:57.302+0000 7f6ca8553640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:27:57.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:57 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a[25336]: 2024-09-16T10:27:57.302+0000 7f6ca8553640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T10:27:57.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:57 smithi164 podman[25674]: 2024-09-16 10:27:57.466605673 +0000 UTC m=+0.275617625 container died ed97711e9992994f68fc81831d1d4c485a4ccb63d7dffd4bd4444e090d7228bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T10:27:57.978 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:57 smithi164 podman[25674]: 2024-09-16 10:27:57.792441323 +0000 UTC m=+0.601453264 container cleanup ed97711e9992994f68fc81831d1d4c485a4ccb63d7dffd4bd4444e090d7228bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.license=GPLv2) 2024-09-16T10:27:57.978 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:57 smithi164 bash[25674]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a 2024-09-16T10:27:58.403 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:57 smithi164 podman[25691]: 2024-09-16 10:27:57.976726351 +0000 UTC m=+0.508156519 container remove ed97711e9992994f68fc81831d1d4c485a4ccb63d7dffd4bd4444e090d7228bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T10:27:58.404 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T10:27:58.404 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 systemd[1]: Stopped Ceph mon.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:27:58.404 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 systemd[1]: Starting Ceph mon.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:27:58.673 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 podman[25790]: 2024-09-16 10:27:58.402899538 +0000 UTC m=+0.110739760 container create 3513d0b825e1dbf0ee0759252d2a89ccf2129e55fd8088c2cf0fc5b7981a942c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T10:27:58.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 podman[25790]: 2024-09-16 10:27:58.30761445 +0000 UTC m=+0.015454672 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T10:27:58.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 podman[25790]: 2024-09-16 10:27:58.588568652 +0000 UTC m=+0.296408874 container init 3513d0b825e1dbf0ee0759252d2a89ccf2129e55fd8088c2cf0fc5b7981a942c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T10:27:58.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 podman[25790]: 2024-09-16 10:27:58.593481918 +0000 UTC m=+0.301322141 container start 3513d0b825e1dbf0ee0759252d2a89ccf2129e55fd8088c2cf0fc5b7981a942c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T10:27:58.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T10:27:58.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable), process ceph-mon, pid 7 2024-09-16T10:27:58.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: pidfile_write: ignore empty --pid-file 2024-09-16T10:27:58.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: load: jerasure load: lrc 2024-09-16T10:27:58.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: RocksDB version: 7.9.2 2024-09-16T10:27:58.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Git sha 0 2024-09-16T10:27:58.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Compile date 2024-09-13 16:45:57 2024-09-16T10:27:58.675 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: DB SUMMARY 2024-09-16T10:27:58.675 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: DB Session ID: VMTAK82MBA6YNDBGO9NF 2024-09-16T10:27:58.675 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: CURRENT file: CURRENT 2024-09-16T10:27:58.675 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T10:27:58.675 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T10:27:58.675 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T10:27:58.675 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 83766 ; 2024-09-16T10:27:58.675 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.error_if_exists: 0 2024-09-16T10:27:58.677 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.create_if_missing: 0 2024-09-16T10:27:58.677 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.paranoid_checks: 1 2024-09-16T10:27:58.677 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T10:27:58.677 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T10:27:58.677 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T10:27:58.677 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.env: 0x56520b691bc0 2024-09-16T10:27:58.677 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T10:27:58.677 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.info_log: 0x56520dd54be0 2024-09-16T10:27:58.677 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.statistics: (nil) 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.use_fsync: 0 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_log_file_size: 0 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.allow_fallocate: 1 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.use_direct_reads: 0 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T10:27:58.678 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T10:27:58.679 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.db_log_dir: 2024-09-16T10:27:58.679 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.wal_dir: 2024-09-16T10:27:58.679 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T10:27:58.679 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T10:27:58.679 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T10:27:58.679 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T10:27:58.679 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T10:27:58.679 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T10:27:58.679 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T10:27:58.679 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T10:27:58.680 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.write_buffer_manager: 0x56520dd62320 2024-09-16T10:27:58.680 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T10:27:58.680 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T10:27:58.680 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T10:27:58.680 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T10:27:58.680 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T10:27:58.680 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T10:27:58.680 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T10:27:58.680 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.unordered_write: 0 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.row_cache: None 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.wal_filter: None 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.two_write_queues: 0 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.wal_compression: 0 2024-09-16T10:27:58.681 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.atomic_flush: 0 2024-09-16T10:27:58.682 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T10:27:58.682 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T10:27:58.682 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T10:27:58.682 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.log_readahead_size: 0 2024-09-16T10:27:58.682 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T10:27:58.682 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T10:27:58.682 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T10:27:58.682 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T10:27:58.682 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T10:27:58.682 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T10:27:58.683 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T10:27:58.683 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_background_jobs: 2 2024-09-16T10:27:58.683 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_background_compactions: -1 2024-09-16T10:27:58.683 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_subcompactions: 1 2024-09-16T10:27:58.683 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T10:27:58.683 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T10:27:58.683 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T10:27:58.683 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T10:27:58.683 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_open_files: -1 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_background_flushes: -1 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Compression algorithms supported: 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: kZSTD supported: 0 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: kXpressCompression supported: 0 2024-09-16T10:27:58.684 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: kBZip2Compression supported: 0 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: kLZ4Compression supported: 1 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: kZlibCompression supported: 1 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: kSnappyCompression supported: 1 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.merge_operator: 2024-09-16T10:27:58.685 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_filter: None 2024-09-16T10:27:58.686 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T10:27:58.686 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T10:27:58.686 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T10:27:58.686 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T10:27:58.686 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56520dd547e0) 2024-09-16T10:27:58.686 INFO:journalctl@ceph.mon.a.smithi164.stdout: cache_index_and_filter_blocks: 1 2024-09-16T10:27:58.686 INFO:journalctl@ceph.mon.a.smithi164.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T10:27:58.686 INFO:journalctl@ceph.mon.a.smithi164.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T10:27:58.686 INFO:journalctl@ceph.mon.a.smithi164.stdout: pin_top_level_index_and_filter: 1 2024-09-16T10:27:58.686 INFO:journalctl@ceph.mon.a.smithi164.stdout: index_type: 0 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: data_block_index_type: 0 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: index_shortening: 1 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: checksum: 4 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: no_block_cache: 0 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_cache: 0x56520dd49350 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_cache_name: BinnedLRUCache 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_cache_options: 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: capacity : 536870912 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: num_shard_bits : 4 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: strict_capacity_limit : 0 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: high_pri_pool_ratio: 0.000 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_cache_compressed: (nil) 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: persistent_cache: (nil) 2024-09-16T10:27:58.687 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_size: 4096 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_size_deviation: 10 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_restart_interval: 16 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: index_block_restart_interval: 1 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: metadata_block_size: 4096 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: partition_filters: 0 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: use_delta_encoding: 1 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: filter_policy: bloomfilter 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: whole_key_filtering: 1 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: verify_compression: 0 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: read_amp_bytes_per_bit: 0 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: format_version: 5 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: enable_index_compression: 1 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_align: 0 2024-09-16T10:27:58.688 INFO:journalctl@ceph.mon.a.smithi164.stdout: max_auto_readahead_size: 262144 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout: prepopulate_block_cache: 0 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout: initial_auto_readahead_size: 8192 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compression: NoCompression 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.num_levels: 7 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T10:27:58.689 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T10:27:58.690 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T10:27:58.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T10:27:58.692 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.inplace_update_support: 0 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T10:27:58.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.bloom_locality: 0 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.max_successive_merges: 0 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.ttl: 2592000 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.enable_blob_files: false 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.min_blob_size: 0 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T10:27:58.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T10:27:58.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T10:27:58.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T10:27:58.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T10:27:58.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T10:27:58.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T10:27:58.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T10:27:58.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T10:27:58.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T10:27:58.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d2d3ee86-ac3b-4864-abe2-0e6f50542154 2024-09-16T10:27:58.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726482478618761, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T10:27:58.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726482478619689, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 80539, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 250, "table_properties": {"data_size": 78700, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9896, "raw_average_key_size": 47, "raw_value_size": 73008, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 209, "num_filter_entries": 209, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726482478, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d2d3ee86-ac3b-4864-abe2-0e6f50542154", "db_session_id": "VMTAK82MBA6YNDBGO9NF", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726482478619793, "job": 1, "event": "recovery_finished"} 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56520dd74e00 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: DB pointer 0x56520de68000 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout: ** DB Stats ** 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T10:27:58.696 INFO:journalctl@ceph.mon.a.smithi164.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T10:27:58.697 INFO:journalctl@ceph.mon.a.smithi164.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T10:27:58.697 INFO:journalctl@ceph.mon.a.smithi164.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T10:27:58.697 INFO:journalctl@ceph.mon.a.smithi164.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T10:27:58.697 INFO:journalctl@ceph.mon.a.smithi164.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T10:27:58.697 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-16T10:27:58.697 INFO:journalctl@ceph.mon.a.smithi164.stdout: ** Compaction Stats [default] ** 2024-09-16T10:27:58.697 INFO:journalctl@ceph.mon.a.smithi164.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T10:27:58.697 INFO:journalctl@ceph.mon.a.smithi164.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T10:27:58.697 INFO:journalctl@ceph.mon.a.smithi164.stdout: L0 2/0 80.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 122.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: Sum 2/0 80.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 122.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 122.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: ** Compaction Stats [default] ** 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 122.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T10:27:58.698 INFO:journalctl@ceph.mon.a.smithi164.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout: Cumulative compaction: 0.00 GB write, 25.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout: Interval compaction: 0.00 GB write, 25.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout: Block cache BinnedLRUCache@0x56520dd49350#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: starting mon.a rank 0 at public addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] at bind addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: mon.a@-1(???) e1 preinit fsid 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: mon.a@-1(???).mds e1 new map 2024-09-16T10:27:58.699 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: mon.a@-1(???).mds e1 print_map 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout: e1 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout: legacy client fscid: -1 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout: No filesystems configured 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 bash[25790]: 3513d0b825e1dbf0ee0759252d2a89ccf2129e55fd8088c2cf0fc5b7981a942c 2024-09-16T10:27:58.700 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 systemd[1]: Started Ceph mon.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:27:58.701 INFO:teuthology.orchestra.run.smithi164.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T10:27:58.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T10:27:58.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: monmap e1: 1 mons at {a=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T10:27:58.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: fsmap 2024-09-16T10:27:58.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T10:27:58.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:27:58 smithi164 ceph-mon[25828]: mgrmap e1: no daemons active 2024-09-16T10:27:59.674 INFO:teuthology.orchestra.run.smithi164.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T10:27:59.695 INFO:teuthology.orchestra.run.smithi164.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T10:27:59.696 INFO:teuthology.orchestra.run.smithi164.stdout:Creating mgr... 2024-09-16T10:27:59.696 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T10:27:59.696 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T10:27:59.948 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mgr.y 2024-09-16T10:27:59.948 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-16T10:28:00.109 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3d5087c4-7416-11ef-bceb-c7b262605968.target.wants/ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-3d5087c4-7416-11ef-bceb-c7b262605968@.service. 2024-09-16T10:28:00.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:00 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/4017695040' entity='client.admin' 2024-09-16T10:28:00.444 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:00 smithi164 podman[26063]: 2024-09-16 10:28:00.247540146 +0000 UTC m=+0.015514697 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T10:28:00.664 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T10:28:00.664 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout disabled 2024-09-16T10:28:00.671 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T10:28:00.671 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-16T10:28:00.671 INFO:teuthology.orchestra.run.smithi164.stdout:firewalld.service is not enabled 2024-09-16T10:28:00.671 INFO:teuthology.orchestra.run.smithi164.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T10:28:00.680 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T10:28:00.680 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout disabled 2024-09-16T10:28:00.687 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T10:28:00.687 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-16T10:28:00.688 INFO:teuthology.orchestra.run.smithi164.stdout:firewalld.service is not enabled 2024-09-16T10:28:00.688 INFO:teuthology.orchestra.run.smithi164.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T10:28:00.688 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mgr to start... 2024-09-16T10:28:00.688 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mgr... 2024-09-16T10:28:00.818 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:00 smithi164 podman[26063]: 2024-09-16 10:28:00.549111033 +0000 UTC m=+0.317085583 container init 3569545997fca00d2f0c2c898dc258cb14b4b796b506dd16e43d7ddda0159296 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0) 2024-09-16T10:28:00.818 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:00 smithi164 podman[26063]: 2024-09-16 10:28:00.553947339 +0000 UTC m=+0.321921883 container start 3569545997fca00d2f0c2c898dc258cb14b4b796b506dd16e43d7ddda0159296 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T10:28:00.818 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:00 smithi164 bash[26063]: 3569545997fca00d2f0c2c898dc258cb14b4b796b506dd16e43d7ddda0159296 2024-09-16T10:28:00.818 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:00 smithi164 systemd[1]: Started Ceph mgr.y for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:28:00.818 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:00 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:00.672+0000 7fd0dc9db040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T10:28:00.818 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:00 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:00.741+0000 7fd0dc9db040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T10:28:01.086 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:00 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:00.889+0000 7fd0dc9db040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T10:28:01.087 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:00 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:00.962+0000 7fd0dc9db040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T10:28:01.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:01 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3420153149' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T10:28:01.444 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:01 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:01.116+0000 7fd0dc9db040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:28:01.444 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:01 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:01.265+0000 7fd0dc9db040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T10:28:01.822 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:01 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:01.579+0000 7fd0dc9db040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T10:28:01.823 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:01 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:01.639+0000 7fd0dc9db040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T10:28:01.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:28:01.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-16T10:28:01.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsid": "3d5087c4-7416-11ef-bceb-c7b262605968", 2024-09-16T10:28:01.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T10:28:01.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T10:28:01.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T10:28:01.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T10:28:01.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:01.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T10:28:01.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 0 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T10:28:01.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T10:28:01.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T10:28:01.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T10:28:01.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T10:28:01.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T10:28:01.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:01.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T10:28:01.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T10:28:01.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T10:28:01.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T10:27:53.772238+0000", 2024-09-16T10:28:01.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T10:28:01.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:01.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T10:28:01.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-16T10:28:01.870 INFO:teuthology.orchestra.run.smithi164.stdout:mgr not available, waiting (1/15)... 2024-09-16T10:28:02.193 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:01 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:01.821+0000 7fd0dc9db040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T10:28:02.943 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:02 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:02.680+0000 7fd0dc9db040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T10:28:02.943 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:02 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:02.744+0000 7fd0dc9db040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T10:28:03.703 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:03 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:03.448+0000 7fd0dc9db040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T10:28:03.704 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:03 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:03.561+0000 7fd0dc9db040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T10:28:03.704 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:03 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:03.701+0000 7fd0dc9db040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:28:03.981 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:03 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:03.762+0000 7fd0dc9db040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T10:28:03.981 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:03 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:03.911+0000 7fd0dc9db040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T10:28:03.981 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:03 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:03.979+0000 7fd0dc9db040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T10:28:04.269 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:04 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:04.040+0000 7fd0dc9db040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T10:28:04.269 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:04 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:04.202+0000 7fd0dc9db040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T10:28:04.269 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:04 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:04.267+0000 7fd0dc9db040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T10:28:04.547 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:04 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/2655148346' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T10:28:04.547 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:04 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:04.336+0000 7fd0dc9db040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T10:28:04.547 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:04 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:04.422+0000 7fd0dc9db040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T10:28:04.547 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:04 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:04.486+0000 7fd0dc9db040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T10:28:04.547 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:04 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:04.545+0000 7fd0dc9db040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T10:28:04.937 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:28:04.937 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-16T10:28:04.937 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsid": "3d5087c4-7416-11ef-bceb-c7b262605968", 2024-09-16T10:28:04.937 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T10:28:04.937 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T10:28:04.937 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T10:28:04.937 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T10:28:04.937 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:04.937 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 0 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:04.938 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T10:28:04.939 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T10:28:04.940 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T10:27:53.772238+0000", 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-16T10:28:04.941 INFO:teuthology.orchestra.run.smithi164.stdout:mgr not available, waiting (2/15)... 2024-09-16T10:28:05.693 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:05.328+0000 7fd0dc9db040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T10:28:05.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: Activating manager daemon y 2024-09-16T10:28:05.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: mgrmap e2: y(active, starting, since 0.00310889s) 2024-09-16T10:28:05.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: from='mgr.14100 172.21.15.164:0/3826265140' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T10:28:05.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: from='mgr.14100 172.21.15.164:0/3826265140' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T10:28:05.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: from='mgr.14100 172.21.15.164:0/3826265140' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T10:28:05.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: from='mgr.14100 172.21.15.164:0/3826265140' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:28:05.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: from='mgr.14100 172.21.15.164:0/3826265140' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T10:28:05.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: Manager daemon y is now available 2024-09-16T10:28:05.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: from='mgr.14100 172.21.15.164:0/3826265140' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T10:28:05.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: from='mgr.14100 172.21.15.164:0/3826265140' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T10:28:05.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: from='mgr.14100 172.21.15.164:0/3826265140' entity='mgr.y' 2024-09-16T10:28:05.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: from='mgr.14100 172.21.15.164:0/3826265140' entity='mgr.y' 2024-09-16T10:28:05.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:05 smithi164 ceph-mon[25828]: from='mgr.14100 172.21.15.164:0/3826265140' entity='mgr.y' 2024-09-16T10:28:07.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:07 smithi164 ceph-mon[25828]: mgrmap e3: y(active, since 1.00693s) 2024-09-16T10:28:08.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:08 smithi164 ceph-mon[25828]: mgrmap e4: y(active, since 2s) 2024-09-16T10:28:09.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:09 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/2341432631' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T10:28:14.651 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:28:14.651 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-16T10:28:14.651 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsid": "3d5087c4-7416-11ef-bceb-c7b262605968", 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 0 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T10:28:14.652 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T10:28:14.653 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T10:28:14.657 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T10:28:14.658 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T10:28:14.659 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-16T10:28:14.660 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T10:28:14.660 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:14.660 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T10:28:14.660 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T10:28:14.660 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T10:27:53.772238+0000", 2024-09-16T10:28:14.660 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T10:28:14.660 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-16T10:28:14.660 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T10:28:14.660 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-16T10:28:14.660 INFO:teuthology.orchestra.run.smithi164.stdout:mgr is available 2024-09-16T10:28:18.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:17 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3312791248' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T10:28:18.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:17 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3312791248' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T10:28:19.056 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:28:19.056 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T10:28:19.056 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout fsid = 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T10:28:19.057 INFO:teuthology.orchestra.run.smithi164.stdout:Enabling cephadm module... 2024-09-16T10:28:20.151 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:19 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3921452254' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T10:28:20.151 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:19 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ignoring --setuser ceph since I am not root 2024-09-16T10:28:20.151 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:19 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ignoring --setgroup ceph since I am not root 2024-09-16T10:28:20.151 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:19 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:19.944+0000 7fd48ab77040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T10:28:20.151 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:20 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:19.999+0000 7fd48ab77040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T10:28:20.151 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:20 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:20.148+0000 7fd48ab77040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T10:28:20.443 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:20 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:20.216+0000 7fd48ab77040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T10:28:20.443 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:20 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:20.358+0000 7fd48ab77040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:28:20.803 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:20 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:20.502+0000 7fd48ab77040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T10:28:21.092 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:20 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3921452254' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T10:28:21.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:20 smithi164 ceph-mon[25828]: mgrmap e5: y(active, since 14s) 2024-09-16T10:28:21.093 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:20 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:20.817+0000 7fd48ab77040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T10:28:21.093 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:20 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:20.879+0000 7fd48ab77040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T10:28:21.094 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:21 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:21.090+0000 7fd48ab77040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T10:28:22.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:21 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/334815822' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T10:28:22.194 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:21 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:21.965+0000 7fd48ab77040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T10:28:22.194 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:22 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:22.029+0000 7fd48ab77040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T10:28:22.236 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-16T10:28:22.237 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T10:28:22.237 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T10:28:22.237 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T10:28:22.237 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T10:28:22.237 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-16T10:28:22.237 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for the mgr to restart... 2024-09-16T10:28:22.237 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mgr epoch 5... 2024-09-16T10:28:23.025 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:22 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:22.752+0000 7fd48ab77040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T10:28:23.025 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:22 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:22.885+0000 7fd48ab77040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T10:28:23.294 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:23 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:23.023+0000 7fd48ab77040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:28:23.294 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:23 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:23.082+0000 7fd48ab77040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T10:28:23.295 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:23 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:23.227+0000 7fd48ab77040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T10:28:23.295 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:23 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:23.292+0000 7fd48ab77040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T10:28:23.577 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:23 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:23.353+0000 7fd48ab77040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T10:28:23.577 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:23 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:23.512+0000 7fd48ab77040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T10:28:23.578 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:23 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:23.575+0000 7fd48ab77040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T10:28:23.839 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:23 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:23.641+0000 7fd48ab77040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T10:28:23.839 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:23 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:23.721+0000 7fd48ab77040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T10:28:23.839 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:23 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:23.779+0000 7fd48ab77040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T10:28:23.839 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:23 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:23.837+0000 7fd48ab77040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T10:28:24.943 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:24.595+0000 7fd48ab77040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T10:28:24.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: Active manager daemon y restarted 2024-09-16T10:28:24.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: Activating manager daemon y 2024-09-16T10:28:24.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T10:28:24.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: mgrmap e6: y(active, starting, since 0.00292863s) 2024-09-16T10:28:24.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:28:24.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T10:28:24.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T10:28:24.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T10:28:24.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T10:28:24.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: Manager daemon y is now available 2024-09-16T10:28:24.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:24.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:24.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:28:24.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:28:24.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T10:28:24.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:24 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T10:28:25.928 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:25 smithi164 ceph-mon[25828]: Found migration_current of "None". Setting to last migration. 2024-09-16T10:28:25.929 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:25 smithi164 ceph-mon[25828]: mgrmap e7: y(active, since 1.00497s) 2024-09-16T10:28:26.211 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-16T10:28:26.211 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T10:28:26.211 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T10:28:26.211 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-16T10:28:26.211 INFO:teuthology.orchestra.run.smithi164.stdout:mgr epoch 5 is available 2024-09-16T10:28:26.211 INFO:teuthology.orchestra.run.smithi164.stdout:Setting orchestrator backend to cephadm... 2024-09-16T10:28:26.843 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-mon[25828]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T10:28:26.843 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-mon[25828]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T10:28:26.843 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:26.843 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:27.145 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:26.841+0000 7fd3f80a2640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:10:28:26] ENGINE Error in HTTPServer.serve 2024-09-16T10:28:27.145 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: Traceback (most recent call last): 2024-09-16T10:28:27.145 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T10:28:27.145 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self._connections.run(self.expiration_interval) 2024-09-16T10:28:27.145 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T10:28:27.145 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self._run(expiration_interval) 2024-09-16T10:28:27.145 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T10:28:27.145 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T10:28:27.145 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T10:28:27.145 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T10:28:27.145 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T10:28:27.146 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: s = self.context.wrap_socket( 2024-09-16T10:28:27.146 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T10:28:27.146 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: return self.sslsocket_class._create( 2024-09-16T10:28:27.146 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T10:28:27.146 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self.do_handshake() 2024-09-16T10:28:27.146 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T10:28:27.146 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self._sslobj.do_handshake() 2024-09-16T10:28:27.146 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T10:28:27.146 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:27.991 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:27 smithi164 ceph-mon[25828]: [16/Sep/2024:10:28:26] ENGINE Bus STARTING 2024-09-16T10:28:27.991 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:27 smithi164 ceph-mon[25828]: [16/Sep/2024:10:28:26] ENGINE Serving on https://172.21.15.164:7150 2024-09-16T10:28:27.991 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:27 smithi164 ceph-mon[25828]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:28:27.991 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:27 smithi164 ceph-mon[25828]: [16/Sep/2024:10:28:26] ENGINE Error in HTTPServer.serve 2024-09-16T10:28:27.991 INFO:journalctl@ceph.mon.a.smithi164.stdout: Traceback (most recent call last): 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._connections.run(self.expiration_interval) 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._run(expiration_interval) 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: s = self.context.wrap_socket( 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T10:28:27.992 INFO:journalctl@ceph.mon.a.smithi164.stdout: return self.sslsocket_class._create( 2024-09-16T10:28:27.993 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T10:28:27.993 INFO:journalctl@ceph.mon.a.smithi164.stdout: self.do_handshake() 2024-09-16T10:28:27.993 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T10:28:27.993 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._sslobj.do_handshake() 2024-09-16T10:28:27.993 INFO:journalctl@ceph.mon.a.smithi164.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T10:28:27.993 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:27 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:27.993 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:27 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:28:27.993 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:27 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:28:27.993 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:27 smithi164 ceph-mon[25828]: mgrmap e8: y(active, since 2s) 2024-09-16T10:28:28.577 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T10:28:28.577 INFO:teuthology.orchestra.run.smithi164.stdout:Generating ssh key... 2024-09-16T10:28:29.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:28 smithi164 ceph-mon[25828]: [16/Sep/2024:10:28:26] ENGINE Serving on http://172.21.15.164:8765 2024-09-16T10:28:29.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:28 smithi164 ceph-mon[25828]: [16/Sep/2024:10:28:26] ENGINE Bus STARTED 2024-09-16T10:28:29.689 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: Generating public/private rsa key pair. 2024-09-16T10:28:29.689 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: Your identification has been saved in /tmp/tmpkzci4nub/key 2024-09-16T10:28:29.689 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: Your public key has been saved in /tmp/tmpkzci4nub/key.pub 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: The key fingerprint is: 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: SHA256:/5kOMtBBGF7LaKniUenzTykHSh/QMgff84oYJY1MpEY ceph-3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: The key's randomart image is: 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: +---[RSA 3072]----+ 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: | E.+ .oo | 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: | . + O.B . | 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: | o X @ * | 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: | . o X . + | 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: | o * + S . | 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: | . + B = + | 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: | . o = B o | 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: | = o o o | 2024-09-16T10:28:29.690 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: | . .= | 2024-09-16T10:28:29.691 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: +----[SHA256]-----+ 2024-09-16T10:28:29.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-mon[25828]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:28:29.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:29.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:29 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:31.121 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:30 smithi164 ceph-mon[25828]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:28:31.121 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:30 smithi164 ceph-mon[25828]: Generating ssh key... 2024-09-16T10:28:31.123 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgoEj03uJhC6UUd81nyUGbY+Wyy24anf12CTAcwlef73VlObep8whaLzJ8e6/dOUTIesTZv2LfaTErkJM/9xQ128GVwPmKoj46c1xYhxvtDhal5xqXtgtRZ5lnm/tVEZlUAT6Tcu73MOxGbrA+7pFcTzmJwyT+MTO5bQRmvsDVAWK3smBrlTEp0UtsyAe8XTN2TzRbwoaSfG28bLhcEDF1N+akSK/qsf1A+pNUf2nkTSaxp6SatgE6MulcV86D0/DtZI3EA/iKjPpmna+mZPfpdZpdMq9xZhlKk0iJocBcb7YbcsFsV8sg0Q6+Gev9SPwmKX05nXq00+PQcyPVC5HdYS77cZfS9killycbLoS/smhFz3ILnFSDvpLhNU4k5LkR1OM6WLQ7sFfIjY72tX8odedRiIX60Ziq0+SIdRdWyNDzUx3wchmYCLgD/yQa3YE4x2nlXRHwHEaZSlPyzGdLgyeBnYYVU9HsDgmhFKA2f0Bwzk/93N2mxQ6D0VHU4xU= ceph-3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:28:31.123 INFO:teuthology.orchestra.run.smithi164.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T10:28:31.123 INFO:teuthology.orchestra.run.smithi164.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T10:28:31.133 INFO:teuthology.orchestra.run.smithi164.stdout:Adding host smithi164... 2024-09-16T10:28:32.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:31 smithi164 ceph-mon[25828]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:28:33.172 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:32 smithi164 ceph-mon[25828]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "addr": "172.21.15.164", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:28:34.063 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:33 smithi164 ceph-mon[25828]: Deploying cephadm binary to smithi164 2024-09-16T10:28:34.956 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout Added host 'smithi164' with addr '172.21.15.164' 2024-09-16T10:28:34.956 INFO:teuthology.orchestra.run.smithi164.stdout:Deploying unmanaged mon service... 2024-09-16T10:28:35.546 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:35 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:35.546 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:35 smithi164 ceph-mon[25828]: Added host smithi164 2024-09-16T10:28:35.546 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:35 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:28:36.933 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:36 smithi164 ceph-mon[25828]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:28:36.933 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:36 smithi164 ceph-mon[25828]: Saving service mon spec with placement count:5 2024-09-16T10:28:36.933 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:36 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:36.933 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:36 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:36.934 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T10:28:36.934 INFO:teuthology.orchestra.run.smithi164.stdout:Deploying unmanaged mgr service... 2024-09-16T10:28:38.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:38 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:38.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:38 smithi164 ceph-mon[25828]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:28:38.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:38 smithi164 ceph-mon[25828]: Saving service mgr spec with placement count:2 2024-09-16T10:28:38.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:38 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:38.880 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T10:28:39.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:39 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:39.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:39 smithi164 ceph-mon[25828]: from='mgr.14118 172.21.15.164:0/996896619' entity='mgr.y' 2024-09-16T10:28:40.805 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:40 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/777075922' entity='client.admin' 2024-09-16T10:28:41.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:41 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/505075146' entity='client.admin' 2024-09-16T10:28:41.998 INFO:teuthology.orchestra.run.smithi164.stdout:Enabling the dashboard module... 2024-09-16T10:28:42.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:42 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1754976762' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T10:28:43.267 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:42 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ignoring --setuser ceph since I am not root 2024-09-16T10:28:43.267 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:42 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ignoring --setgroup ceph since I am not root 2024-09-16T10:28:43.267 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:43 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:43.055+0000 7fdbc8a08040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T10:28:43.267 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:43 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:43.112+0000 7fdbc8a08040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T10:28:43.267 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:43 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:43.264+0000 7fdbc8a08040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T10:28:43.586 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:43 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:43.332+0000 7fdbc8a08040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T10:28:43.586 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:43 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:43.481+0000 7fdbc8a08040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:28:43.842 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:43 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:43.629+0000 7fdbc8a08040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T10:28:44.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:43 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1754976762' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T10:28:44.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:43 smithi164 ceph-mon[25828]: mgrmap e9: y(active, since 18s) 2024-09-16T10:28:44.194 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:43 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:43.944+0000 7fdbc8a08040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T10:28:44.194 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:44.008+0000 7fdbc8a08040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T10:28:44.574 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:44.198+0000 7fdbc8a08040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T10:28:44.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-16T10:28:44.880 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T10:28:44.880 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T10:28:44.880 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T10:28:44.880 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T10:28:44.880 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-16T10:28:44.880 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for the mgr to restart... 2024-09-16T10:28:44.880 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mgr epoch 9... 2024-09-16T10:28:45.161 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:44 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/615155519' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T10:28:45.161 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:45 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:45.089+0000 7fdbc8a08040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T10:28:45.443 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:45 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:45.158+0000 7fdbc8a08040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T10:28:46.120 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:45 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:45.866+0000 7fdbc8a08040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T10:28:46.120 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:45 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:45.979+0000 7fdbc8a08040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T10:28:46.391 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:46.117+0000 7fdbc8a08040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:28:46.391 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:46.178+0000 7fdbc8a08040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T10:28:46.391 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:46.322+0000 7fdbc8a08040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T10:28:46.392 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:46.388+0000 7fdbc8a08040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T10:28:46.669 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:46.447+0000 7fdbc8a08040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T10:28:46.669 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:46.602+0000 7fdbc8a08040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T10:28:46.669 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:46.666+0000 7fdbc8a08040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T10:28:46.931 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:46.732+0000 7fdbc8a08040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T10:28:46.931 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:46.812+0000 7fdbc8a08040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T10:28:46.931 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:46.870+0000 7fdbc8a08040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T10:28:46.931 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:46.928+0000 7fdbc8a08040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T10:28:48.193 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:47.693+0000 7fdbc8a08040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T10:28:48.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: Active manager daemon y restarted 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: Activating manager daemon y 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: mgrmap e10: y(active, starting, since 0.00306524s) 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: Manager daemon y is now available 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T10:28:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T10:28:48.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:48.673+0000 7fdb3469d640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:10:28:48] ENGINE Error in HTTPServer.serve 2024-09-16T10:28:48.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: Traceback (most recent call last): 2024-09-16T10:28:48.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T10:28:48.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self._connections.run(self.expiration_interval) 2024-09-16T10:28:48.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T10:28:48.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self._run(expiration_interval) 2024-09-16T10:28:48.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T10:28:48.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T10:28:48.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T10:28:48.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T10:28:48.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T10:28:48.710 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: s = self.context.wrap_socket( 2024-09-16T10:28:48.710 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T10:28:48.710 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: return self.sslsocket_class._create( 2024-09-16T10:28:48.710 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T10:28:48.710 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self.do_handshake() 2024-09-16T10:28:48.710 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T10:28:48.710 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self._sslobj.do_handshake() 2024-09-16T10:28:48.710 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T10:28:48.710 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:28:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:28:49.487 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-16T10:28:49.487 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T10:28:49.487 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T10:28:49.487 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-16T10:28:49.487 INFO:teuthology.orchestra.run.smithi164.stdout:mgr epoch 9 is available 2024-09-16T10:28:49.488 INFO:teuthology.orchestra.run.smithi164.stdout:Generating a dashboard self-signed certificate... 2024-09-16T10:28:49.740 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:49 smithi164 ceph-mon[25828]: [16/Sep/2024:10:28:48] ENGINE Bus STARTING 2024-09-16T10:28:49.740 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:49 smithi164 ceph-mon[25828]: [16/Sep/2024:10:28:48] ENGINE Serving on https://172.21.15.164:7150 2024-09-16T10:28:49.740 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:49 smithi164 ceph-mon[25828]: [16/Sep/2024:10:28:48] ENGINE Error in HTTPServer.serve 2024-09-16T10:28:49.740 INFO:journalctl@ceph.mon.a.smithi164.stdout: Traceback (most recent call last): 2024-09-16T10:28:49.740 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T10:28:49.741 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._connections.run(self.expiration_interval) 2024-09-16T10:28:49.741 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T10:28:49.741 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._run(expiration_interval) 2024-09-16T10:28:49.741 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T10:28:49.741 INFO:journalctl@ceph.mon.a.smithi164.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T10:28:49.741 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T10:28:49.741 INFO:journalctl@ceph.mon.a.smithi164.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T10:28:49.741 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T10:28:49.741 INFO:journalctl@ceph.mon.a.smithi164.stdout: s = self.context.wrap_socket( 2024-09-16T10:28:49.741 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T10:28:49.741 INFO:journalctl@ceph.mon.a.smithi164.stdout: return self.sslsocket_class._create( 2024-09-16T10:28:49.742 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T10:28:49.742 INFO:journalctl@ceph.mon.a.smithi164.stdout: self.do_handshake() 2024-09-16T10:28:49.742 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T10:28:49.742 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._sslobj.do_handshake() 2024-09-16T10:28:49.742 INFO:journalctl@ceph.mon.a.smithi164.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T10:28:49.742 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:49 smithi164 ceph-mon[25828]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T10:28:49.742 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:49 smithi164 ceph-mon[25828]: mgrmap e11: y(active, since 1.00533s) 2024-09-16T10:28:49.742 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:49 smithi164 ceph-mon[25828]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T10:28:49.742 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:49 smithi164 ceph-mon[25828]: [16/Sep/2024:10:28:48] ENGINE Serving on http://172.21.15.164:8765 2024-09-16T10:28:49.742 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:49 smithi164 ceph-mon[25828]: [16/Sep/2024:10:28:48] ENGINE Bus STARTED 2024-09-16T10:28:49.742 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:49 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:28:50.768 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T10:28:50.768 INFO:teuthology.orchestra.run.smithi164.stdout:Creating initial admin user... 2024-09-16T10:28:51.394 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:51 smithi164 ceph-mon[25828]: mgrmap e12: y(active, since 2s) 2024-09-16T10:28:51.394 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:51 smithi164 ceph-mon[25828]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:28:51.395 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:51 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:28:51.395 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:51 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:28:51.395 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:51 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:28:52.713 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$LKnk3.jzdq5M4PGGobdskOoyFKM2me.ltDrlTcuJl8Rt2IVhtxxPG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726482531, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T10:28:52.713 INFO:teuthology.orchestra.run.smithi164.stdout:Fetching dashboard port number... 2024-09-16T10:28:52.838 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:52 smithi164 ceph-mon[25828]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:28:52.838 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:52 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:28:53.916 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:53 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/276299278' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T10:28:53.918 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T10:28:53.928 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T10:28:53.928 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout disabled 2024-09-16T10:28:53.935 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T10:28:53.935 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-16T10:28:53.936 INFO:teuthology.orchestra.run.smithi164.stdout:firewalld.service is not enabled 2024-09-16T10:28:53.936 INFO:teuthology.orchestra.run.smithi164.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T10:28:53.937 INFO:teuthology.orchestra.run.smithi164.stdout:Ceph Dashboard is now available at: 2024-09-16T10:28:53.937 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:28:53.937 INFO:teuthology.orchestra.run.smithi164.stdout: URL: https://smithi164.front.sepia.ceph.com:8443/ 2024-09-16T10:28:53.937 INFO:teuthology.orchestra.run.smithi164.stdout: User: admin 2024-09-16T10:28:53.937 INFO:teuthology.orchestra.run.smithi164.stdout: Password: 8qz6s61v6t 2024-09-16T10:28:53.937 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:28:53.938 INFO:teuthology.orchestra.run.smithi164.stdout:Saving cluster configuration to /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config directory 2024-09-16T10:28:53.938 INFO:teuthology.orchestra.run.smithi164.stdout:Enabling autotune for osd_memory_target 2024-09-16T10:28:55.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:55 smithi164 ceph-mon[25828]: mgrmap e13: y(active, since 6s) 2024-09-16T10:28:56.289 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T10:28:56.289 INFO:teuthology.orchestra.run.smithi164.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T10:28:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:28:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T10:28:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:28:56.289 INFO:teuthology.orchestra.run.smithi164.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T10:28:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:28:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T10:28:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:28:56.289 INFO:teuthology.orchestra.run.smithi164.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T10:28:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:28:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: ceph telemetry on 2024-09-16T10:28:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:28:56.290 INFO:teuthology.orchestra.run.smithi164.stdout:For more information see: 2024-09-16T10:28:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:28:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T10:28:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:28:56.290 INFO:teuthology.orchestra.run.smithi164.stdout:Bootstrap complete. 2024-09-16T10:28:56.325 INFO:tasks.cephadm:Fetching config... 2024-09-16T10:28:56.326 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:28:56.326 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T10:28:56.342 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T10:28:56.342 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:28:56.342 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T10:28:56.398 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T10:28:56.399 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:28:56.399 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T10:28:56.464 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T10:28:56.464 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:28:56.465 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T10:28:56.519 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T10:28:56.520 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgoEj03uJhC6UUd81nyUGbY+Wyy24anf12CTAcwlef73VlObep8whaLzJ8e6/dOUTIesTZv2LfaTErkJM/9xQ128GVwPmKoj46c1xYhxvtDhal5xqXtgtRZ5lnm/tVEZlUAT6Tcu73MOxGbrA+7pFcTzmJwyT+MTO5bQRmvsDVAWK3smBrlTEp0UtsyAe8XTN2TzRbwoaSfG28bLhcEDF1N+akSK/qsf1A+pNUf2nkTSaxp6SatgE6MulcV86D0/DtZI3EA/iKjPpmna+mZPfpdZpdMq9xZhlKk0iJocBcb7YbcsFsV8sg0Q6+Gev9SPwmKX05nXq00+PQcyPVC5HdYS77cZfS9killycbLoS/smhFz3ILnFSDvpLhNU4k5LkR1OM6WLQ7sFfIjY72tX8odedRiIX60Ziq0+SIdRdWyNDzUx3wchmYCLgD/yQa3YE4x2nlXRHwHEaZSlPyzGdLgyeBnYYVU9HsDgmhFKA2f0Bwzk/93N2mxQ6D0VHU4xU= ceph-3d5087c4-7416-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T10:28:56.596 INFO:teuthology.orchestra.run.smithi164.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgoEj03uJhC6UUd81nyUGbY+Wyy24anf12CTAcwlef73VlObep8whaLzJ8e6/dOUTIesTZv2LfaTErkJM/9xQ128GVwPmKoj46c1xYhxvtDhal5xqXtgtRZ5lnm/tVEZlUAT6Tcu73MOxGbrA+7pFcTzmJwyT+MTO5bQRmvsDVAWK3smBrlTEp0UtsyAe8XTN2TzRbwoaSfG28bLhcEDF1N+akSK/qsf1A+pNUf2nkTSaxp6SatgE6MulcV86D0/DtZI3EA/iKjPpmna+mZPfpdZpdMq9xZhlKk0iJocBcb7YbcsFsV8sg0Q6+Gev9SPwmKX05nXq00+PQcyPVC5HdYS77cZfS9killycbLoS/smhFz3ILnFSDvpLhNU4k5LkR1OM6WLQ7sFfIjY72tX8odedRiIX60Ziq0+SIdRdWyNDzUx3wchmYCLgD/yQa3YE4x2nlXRHwHEaZSlPyzGdLgyeBnYYVU9HsDgmhFKA2f0Bwzk/93N2mxQ6D0VHU4xU= ceph-3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:28:56.619 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgoEj03uJhC6UUd81nyUGbY+Wyy24anf12CTAcwlef73VlObep8whaLzJ8e6/dOUTIesTZv2LfaTErkJM/9xQ128GVwPmKoj46c1xYhxvtDhal5xqXtgtRZ5lnm/tVEZlUAT6Tcu73MOxGbrA+7pFcTzmJwyT+MTO5bQRmvsDVAWK3smBrlTEp0UtsyAe8XTN2TzRbwoaSfG28bLhcEDF1N+akSK/qsf1A+pNUf2nkTSaxp6SatgE6MulcV86D0/DtZI3EA/iKjPpmna+mZPfpdZpdMq9xZhlKk0iJocBcb7YbcsFsV8sg0Q6+Gev9SPwmKX05nXq00+PQcyPVC5HdYS77cZfS9killycbLoS/smhFz3ILnFSDvpLhNU4k5LkR1OM6WLQ7sFfIjY72tX8odedRiIX60Ziq0+SIdRdWyNDzUx3wchmYCLgD/yQa3YE4x2nlXRHwHEaZSlPyzGdLgyeBnYYVU9HsDgmhFKA2f0Bwzk/93N2mxQ6D0VHU4xU= ceph-3d5087c4-7416-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T10:28:56.677 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgoEj03uJhC6UUd81nyUGbY+Wyy24anf12CTAcwlef73VlObep8whaLzJ8e6/dOUTIesTZv2LfaTErkJM/9xQ128GVwPmKoj46c1xYhxvtDhal5xqXtgtRZ5lnm/tVEZlUAT6Tcu73MOxGbrA+7pFcTzmJwyT+MTO5bQRmvsDVAWK3smBrlTEp0UtsyAe8XTN2TzRbwoaSfG28bLhcEDF1N+akSK/qsf1A+pNUf2nkTSaxp6SatgE6MulcV86D0/DtZI3EA/iKjPpmna+mZPfpdZpdMq9xZhlKk0iJocBcb7YbcsFsV8sg0Q6+Gev9SPwmKX05nXq00+PQcyPVC5HdYS77cZfS9killycbLoS/smhFz3ILnFSDvpLhNU4k5LkR1OM6WLQ7sFfIjY72tX8odedRiIX60Ziq0+SIdRdWyNDzUx3wchmYCLgD/yQa3YE4x2nlXRHwHEaZSlPyzGdLgyeBnYYVU9HsDgmhFKA2f0Bwzk/93N2mxQ6D0VHU4xU= ceph-3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:28:56.690 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T10:28:56.857 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:56 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/209439180' entity='client.admin' 2024-09-16T10:28:56.918 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:28:59.639 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:28:59.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:28:59.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:28:59.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:28:59.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:28:59.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:28:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:00.193 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T10:29:00.194 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T10:29:00.411 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:29:00.426 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:00 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/15449259' entity='client.admin' 2024-09-16T10:29:00.427 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:00 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:00.427 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:00 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:03.491 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi175 2024-09-16T10:29:03.491 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:29:03.491 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.conf 2024-09-16T10:29:03.511 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:29:03.511 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:29:03.570 INFO:tasks.cephadm:Adding host smithi175 to orchestrator... 2024-09-16T10:29:03.570 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch host add smithi175 2024-09-16T10:29:03.760 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:29:03.880 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:03 smithi164 ceph-mon[25828]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:29:03.880 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:03.880 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:03.880 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:03.880 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:03.880 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:03 smithi164 ceph-mon[25828]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T10:29:03.880 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:03 smithi164 ceph-mon[25828]: Updating smithi164:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:29:03.880 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:03.880 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:03.880 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:04.883 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:04 smithi164 ceph-mon[25828]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:29:04.883 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:04 smithi164 ceph-mon[25828]: Updating smithi164:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T10:29:06.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:06 smithi164 ceph-mon[25828]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:29:07.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:07 smithi164 ceph-mon[25828]: Deploying cephadm binary to smithi175 2024-09-16T10:29:08.081 INFO:teuthology.orchestra.run.smithi164.stdout:Added host 'smithi175' with addr '172.21.15.175' 2024-09-16T10:29:08.644 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T10:29:08.835 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:29:08.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:08 smithi164 ceph-mon[25828]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:08.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:08.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:08.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:08.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:09.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:09 smithi164 ceph-mon[25828]: Added host smithi175 2024-09-16T10:29:09.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:09 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:10.630 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:29:10.631 INFO:teuthology.orchestra.run.smithi164.stdout:[{"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}, {"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}] 2024-09-16T10:29:10.934 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:10 smithi164 ceph-mon[25828]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:11.230 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T10:29:11.230 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T10:29:11.418 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:29:11.854 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:11 smithi164 ceph-mon[25828]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T10:29:12.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:12 smithi164 ceph-mon[25828]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:12.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:12.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:12.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:12.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:12.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:29:12.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:12.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:14.024 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:13 smithi164 ceph-mon[25828]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-16T10:29:14.024 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:13 smithi164 ceph-mon[25828]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:29:14.024 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:13 smithi164 ceph-mon[25828]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:29:14.024 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:13 smithi164 ceph-mon[25828]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T10:29:14.025 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:13 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:14.025 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:13 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:14.025 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:13 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:14.025 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:13 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3272425435' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T10:29:14.025 INFO:teuthology.orchestra.run.smithi164.stderr:adjusted tunables profile to default 2024-09-16T10:29:14.601 INFO:tasks.cephadm:Adding mon.a on smithi164 2024-09-16T10:29:14.602 INFO:tasks.cephadm:Adding mon.c on smithi164 2024-09-16T10:29:14.602 INFO:tasks.cephadm:Adding mon.b on smithi175 2024-09-16T10:29:14.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi164:172.21.15.164=a;smithi164:[v2:172.21.15.164:3301,v1:172.21.15.164:6790]=c;smithi175:172.21.15.175=b' 2024-09-16T10:29:14.796 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:29:14.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:14 smithi164 ceph-mon[25828]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:14.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:14 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3272425435' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T10:29:14.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:14 smithi164 ceph-mon[25828]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:16.442 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mon update... 2024-09-16T10:29:17.002 DEBUG:teuthology.orchestra.run.smithi164:mon.c> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T10:29:17.005 DEBUG:teuthology.orchestra.run.smithi175:mon.b> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T10:29:17.007 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T10:29:17.007 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T10:29:17.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:16 smithi164 ceph-mon[25828]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:17.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:16 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:17.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:16 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:17.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:16 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:17.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:16 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:17.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:16 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:17.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:16 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:17.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:16 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:17.625 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:29:18.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:17 smithi164 ceph-mon[25828]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi164:172.21.15.164=a;smithi164:[v2:172.21.15.164:3301,v1:172.21.15.164:6790]=c;smithi175:172.21.15.175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:29:18.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:17 smithi164 ceph-mon[25828]: Saving service mon spec with placement smithi164:172.21.15.164=a;smithi164:[v2:172.21.15.164:3301,v1:172.21.15.164:6790]=c;smithi175:172.21.15.175=b;count:3 2024-09-16T10:29:18.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:17 smithi164 ceph-mon[25828]: Deploying daemon mon.b on smithi175 2024-09-16T10:29:19.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:18 smithi164 ceph-mon[25828]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:19.716 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:29:19.716 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"3d5087c4-7416-11ef-bceb-c7b262605968","modified":"2024-09-16T10:27:50.856202Z","created":"2024-09-16T10:27:50.856202Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T10:29:19.717 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2024-09-16T10:29:20.197 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:19 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/2977708483' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T10:29:21.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:20 smithi164 ceph-mon[25828]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:21.781 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T10:29:21.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T10:29:21.891 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 podman[30285]: 2024-09-16 10:29:21.868075183 +0000 UTC m=+0.351941779 container start 325d79bb3a2ed5de9d7fd1d103b06ad3498181e65f3d22d89c2c64fe0bc164f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T10:29:22.043 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:29:22.164 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T10:29:22.164 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable), process ceph-mon, pid 7 2024-09-16T10:29:22.164 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: pidfile_write: ignore empty --pid-file 2024-09-16T10:29:22.164 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: load: jerasure load: lrc 2024-09-16T10:29:22.164 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: RocksDB version: 7.9.2 2024-09-16T10:29:22.165 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Git sha 0 2024-09-16T10:29:22.165 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Compile date 2024-09-13 16:45:57 2024-09-16T10:29:22.165 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: DB SUMMARY 2024-09-16T10:29:22.165 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: DB Session ID: UZN5GAWGSDZM0MO8UE0V 2024-09-16T10:29:22.165 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: CURRENT file: CURRENT 2024-09-16T10:29:22.165 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T10:29:22.165 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T10:29:22.165 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-16T10:29:22.166 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-16T10:29:22.166 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.error_if_exists: 0 2024-09-16T10:29:22.166 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.create_if_missing: 0 2024-09-16T10:29:22.166 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.paranoid_checks: 1 2024-09-16T10:29:22.166 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T10:29:22.166 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T10:29:22.166 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T10:29:22.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.env: 0x561bbe8eebc0 2024-09-16T10:29:22.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T10:29:22.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.info_log: 0x561bbf3febe0 2024-09-16T10:29:22.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T10:29:22.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.statistics: (nil) 2024-09-16T10:29:22.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.use_fsync: 0 2024-09-16T10:29:22.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_log_file_size: 0 2024-09-16T10:29:22.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T10:29:22.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T10:29:22.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T10:29:22.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T10:29:22.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.allow_fallocate: 1 2024-09-16T10:29:22.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T10:29:22.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T10:29:22.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.use_direct_reads: 0 2024-09-16T10:29:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T10:29:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T10:29:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.db_log_dir: 2024-09-16T10:29:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.wal_dir: 2024-09-16T10:29:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T10:29:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T10:29:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T10:29:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T10:29:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T10:29:22.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.write_buffer_manager: 0x561bbf40c320 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.unordered_write: 0 2024-09-16T10:29:22.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.row_cache: None 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.wal_filter: None 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.two_write_queues: 0 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.wal_compression: 0 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.atomic_flush: 0 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T10:29:22.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T10:29:22.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T10:29:22.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.log_readahead_size: 0 2024-09-16T10:29:22.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T10:29:22.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T10:29:22.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T10:29:22.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T10:29:22.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T10:29:22.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T10:29:22.172 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T10:29:22.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_background_jobs: 2 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_background_compactions: -1 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_subcompactions: 1 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_open_files: -1 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T10:29:22.174 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_background_flushes: -1 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Compression algorithms supported: 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: kZSTD supported: 0 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: kXpressCompression supported: 0 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: kBZip2Compression supported: 0 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: kLZ4Compression supported: 1 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: kZlibCompression supported: 1 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: kSnappyCompression supported: 1 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T10:29:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.merge_operator: 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_filter: None 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561bbf3fe7e0) 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout: cache_index_and_filter_blocks: 1 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout: pin_top_level_index_and_filter: 1 2024-09-16T10:29:22.176 INFO:journalctl@ceph.mon.b.smithi175.stdout: index_type: 0 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: data_block_index_type: 0 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: index_shortening: 1 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: checksum: 4 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: no_block_cache: 0 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache: 0x561bbf3f3350 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache_name: BinnedLRUCache 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache_options: 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: capacity : 536870912 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: num_shard_bits : 4 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: strict_capacity_limit : 0 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: high_pri_pool_ratio: 0.000 2024-09-16T10:29:22.177 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache_compressed: (nil) 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: persistent_cache: (nil) 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_size: 4096 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_size_deviation: 10 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_restart_interval: 16 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: index_block_restart_interval: 1 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: metadata_block_size: 4096 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: partition_filters: 0 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: use_delta_encoding: 1 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: filter_policy: bloomfilter 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: whole_key_filtering: 1 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: verify_compression: 0 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: read_amp_bytes_per_bit: 0 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: format_version: 5 2024-09-16T10:29:22.178 INFO:journalctl@ceph.mon.b.smithi175.stdout: enable_index_compression: 1 2024-09-16T10:29:22.179 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_align: 0 2024-09-16T10:29:22.179 INFO:journalctl@ceph.mon.b.smithi175.stdout: max_auto_readahead_size: 262144 2024-09-16T10:29:22.179 INFO:journalctl@ceph.mon.b.smithi175.stdout: prepopulate_block_cache: 0 2024-09-16T10:29:22.179 INFO:journalctl@ceph.mon.b.smithi175.stdout: initial_auto_readahead_size: 8192 2024-09-16T10:29:22.179 INFO:journalctl@ceph.mon.b.smithi175.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T10:29:22.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T10:29:22.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T10:29:22.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compression: NoCompression 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.num_levels: 7 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T10:29:22.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T10:29:22.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T10:29:22.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T10:29:22.183 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T10:29:22.184 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T10:29:22.184 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T10:29:22.184 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T10:29:22.184 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T10:29:22.184 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T10:29:22.184 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T10:29:22.184 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.inplace_update_support: 0 2024-09-16T10:29:22.184 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T10:29:22.184 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T10:29:22.184 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T10:29:22.184 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.bloom_locality: 0 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.max_successive_merges: 0 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.ttl: 2592000 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.enable_blob_files: false 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.min_blob_size: 0 2024-09-16T10:29:22.185 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0c3c9710-a6bb-412b-87b5-a5846d9a59eb 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726482561895686, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726482561896092, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726482561, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0c3c9710-a6bb-412b-87b5-a5846d9a59eb", "db_session_id": "UZN5GAWGSDZM0MO8UE0V", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T10:29:22.186 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726482561896188, "job": 1, "event": "recovery_finished"} 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561bbf41ee00 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: DB pointer 0x561bbf512000 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** DB Stats ** 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T10:29:22.187 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** Compaction Stats [default] ** 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** Compaction Stats [default] ** 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-16T10:29:22.188 INFO:journalctl@ceph.mon.b.smithi175.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: Block cache BinnedLRUCache@0x561bbf3f3350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T10:29:22.189 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: using public_addr v2:172.21.15.175:0/0 -> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: starting mon.b rank -1 at public addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] at bind addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(???) e0 preinit fsid 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).mds e1 new map 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout: e1 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout: legacy client fscid: -1 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout: No filesystems configured 2024-09-16T10:29:22.190 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:22.191 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/3272425435' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T10:29:22.192 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:22.193 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/3272425435' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T10:29:22.193 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:22.193 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:22.193 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:22.193 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:22.193 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:22.193 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:22.193 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:22.193 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:22.193 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:22.197 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi164:172.21.15.164=a;smithi164:[v2:172.21.15.164:3301,v1:172.21.15.164:6790]=c;smithi175:172.21.15.175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:29:22.197 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: Saving service mon spec with placement smithi164:172.21.15.164=a;smithi164:[v2:172.21.15.164:3301,v1:172.21.15.164:6790]=c;smithi175:172.21.15.175=b;count:3 2024-09-16T10:29:22.197 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: Deploying daemon mon.b on smithi175 2024-09-16T10:29:22.197 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:22.197 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/2977708483' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T10:29:22.197 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:22.198 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 ceph-mon[30341]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T10:29:22.198 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 bash[30285]: 325d79bb3a2ed5de9d7fd1d103b06ad3498181e65f3d22d89c2c64fe0bc164f9 2024-09-16T10:29:22.198 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:21 smithi175 systemd[1]: Started Ceph mon.b for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:29:27.322 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:27.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:29:27.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: mon.a calling monitor election 2024-09-16T10:29:27.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:27.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: mon.b calling monitor election 2024-09-16T10:29:27.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:27.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: monmap e2: 2 mons at {a=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: fsmap 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: mgrmap e13: y(active, since 39s) 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: overall HEALTH_OK 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:27.324 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:26 smithi164 ceph-mon[25828]: Deploying daemon mon.c on smithi164 2024-09-16T10:29:27.340 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:27.340 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:29:27.340 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.340 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: mon.a calling monitor election 2024-09-16T10:29:27.340 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: mon.b calling monitor election 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: monmap e2: 2 mons at {a=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: fsmap 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: mgrmap e13: y(active, since 39s) 2024-09-16T10:29:27.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: overall HEALTH_OK 2024-09-16T10:29:27.342 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:27.342 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:27.342 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:27.342 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:27.342 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:27.342 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:26 smithi175 ceph-mon[30341]: Deploying daemon mon.c on smithi164 2024-09-16T10:29:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:29:27.382 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":2,"fsid":"3d5087c4-7416-11ef-bceb-c7b262605968","modified":"2024-09-16T10:29:21.927339Z","created":"2024-09-16T10:27:50.856202Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T10:29:27.382 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 2 2024-09-16T10:29:28.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:28 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/1015729795' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T10:29:28.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:28 smithi175 ceph-mon[30341]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:28.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:28 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:28.277 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:28 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/1015729795' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T10:29:28.277 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:28 smithi164 ceph-mon[25828]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:28.277 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:28 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:28.971 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T10:29:28.971 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T10:29:29.165 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:29:29.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:29 smithi164 systemd[1]: Starting Ceph mon.c for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:29:30.227 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:29 smithi164 podman[40308]: 2024-09-16 10:29:29.954861761 +0000 UTC m=+0.091199739 container create 35e6e621943d6ca51c68140c0ec6923d0dd32de14dbe956cae468679efa8d471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b) 2024-09-16T10:29:30.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:29 smithi164 podman[40308]: 2024-09-16 10:29:29.878949046 +0000 UTC m=+0.015286985 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T10:29:30.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 podman[40308]: 2024-09-16 10:29:30.169525156 +0000 UTC m=+0.305863088 container init 35e6e621943d6ca51c68140c0ec6923d0dd32de14dbe956cae468679efa8d471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T10:29:30.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 podman[40308]: 2024-09-16 10:29:30.174666545 +0000 UTC m=+0.311004477 container start 35e6e621943d6ca51c68140c0ec6923d0dd32de14dbe956cae468679efa8d471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T10:29:30.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T10:29:30.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable), process ceph-mon, pid 7 2024-09-16T10:29:30.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: pidfile_write: ignore empty --pid-file 2024-09-16T10:29:30.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: load: jerasure load: lrc 2024-09-16T10:29:30.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: RocksDB version: 7.9.2 2024-09-16T10:29:30.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Git sha 0 2024-09-16T10:29:30.228 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Compile date 2024-09-13 16:45:57 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: DB SUMMARY 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: DB Session ID: J9L3EBPT7Q81W82XPMSV 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: CURRENT file: CURRENT 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.error_if_exists: 0 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.create_if_missing: 0 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.paranoid_checks: 1 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T10:29:30.229 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.env: 0x55e039496bc0 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.info_log: 0x55e03a864be0 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.statistics: (nil) 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.use_fsync: 0 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_log_file_size: 0 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T10:29:30.230 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.allow_fallocate: 1 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.use_direct_reads: 0 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.db_log_dir: 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.wal_dir: 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T10:29:30.231 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T10:29:30.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T10:29:30.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T10:29:30.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.write_buffer_manager: 0x55e03a872320 2024-09-16T10:29:30.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T10:29:30.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T10:29:30.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T10:29:30.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T10:29:30.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T10:29:30.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T10:29:30.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T10:29:30.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T10:29:30.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.unordered_write: 0 2024-09-16T10:29:30.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T10:29:30.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T10:29:30.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T10:29:30.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T10:29:30.234 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.row_cache: None 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.wal_filter: None 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.two_write_queues: 0 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.wal_compression: 0 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.atomic_flush: 0 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.log_readahead_size: 0 2024-09-16T10:29:30.235 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_background_jobs: 2 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_background_compactions: -1 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_subcompactions: 1 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T10:29:30.236 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_open_files: -1 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_background_flushes: -1 2024-09-16T10:29:30.237 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Compression algorithms supported: 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: kZSTD supported: 0 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: kXpressCompression supported: 0 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: kBZip2Compression supported: 0 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: kLZ4Compression supported: 1 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: kZlibCompression supported: 1 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: kSnappyCompression supported: 1 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T10:29:30.238 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T10:29:30.239 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.merge_operator: 2024-09-16T10:29:30.239 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_filter: None 2024-09-16T10:29:30.239 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T10:29:30.239 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T10:29:30.239 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T10:29:30.239 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T10:29:30.239 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e03a8647e0) 2024-09-16T10:29:30.239 INFO:journalctl@ceph.mon.c.smithi164.stdout: cache_index_and_filter_blocks: 1 2024-09-16T10:29:30.239 INFO:journalctl@ceph.mon.c.smithi164.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T10:29:30.240 INFO:journalctl@ceph.mon.c.smithi164.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T10:29:30.240 INFO:journalctl@ceph.mon.c.smithi164.stdout: pin_top_level_index_and_filter: 1 2024-09-16T10:29:30.240 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_type: 0 2024-09-16T10:29:30.240 INFO:journalctl@ceph.mon.c.smithi164.stdout: data_block_index_type: 0 2024-09-16T10:29:30.240 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_shortening: 1 2024-09-16T10:29:30.240 INFO:journalctl@ceph.mon.c.smithi164.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T10:29:30.240 INFO:journalctl@ceph.mon.c.smithi164.stdout: checksum: 4 2024-09-16T10:29:30.240 INFO:journalctl@ceph.mon.c.smithi164.stdout: no_block_cache: 0 2024-09-16T10:29:30.240 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache: 0x55e03a8591f0 2024-09-16T10:29:30.241 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_name: BinnedLRUCache 2024-09-16T10:29:30.241 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_options: 2024-09-16T10:29:30.241 INFO:journalctl@ceph.mon.c.smithi164.stdout: capacity : 536870912 2024-09-16T10:29:30.241 INFO:journalctl@ceph.mon.c.smithi164.stdout: num_shard_bits : 4 2024-09-16T10:29:30.241 INFO:journalctl@ceph.mon.c.smithi164.stdout: strict_capacity_limit : 0 2024-09-16T10:29:30.241 INFO:journalctl@ceph.mon.c.smithi164.stdout: high_pri_pool_ratio: 0.000 2024-09-16T10:29:30.241 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_compressed: (nil) 2024-09-16T10:29:30.241 INFO:journalctl@ceph.mon.c.smithi164.stdout: persistent_cache: (nil) 2024-09-16T10:29:30.241 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_size: 4096 2024-09-16T10:29:30.241 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_size_deviation: 10 2024-09-16T10:29:30.242 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_restart_interval: 16 2024-09-16T10:29:30.242 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_block_restart_interval: 1 2024-09-16T10:29:30.242 INFO:journalctl@ceph.mon.c.smithi164.stdout: metadata_block_size: 4096 2024-09-16T10:29:30.242 INFO:journalctl@ceph.mon.c.smithi164.stdout: partition_filters: 0 2024-09-16T10:29:30.242 INFO:journalctl@ceph.mon.c.smithi164.stdout: use_delta_encoding: 1 2024-09-16T10:29:30.242 INFO:journalctl@ceph.mon.c.smithi164.stdout: filter_policy: bloomfilter 2024-09-16T10:29:30.242 INFO:journalctl@ceph.mon.c.smithi164.stdout: whole_key_filtering: 1 2024-09-16T10:29:30.242 INFO:journalctl@ceph.mon.c.smithi164.stdout: verify_compression: 0 2024-09-16T10:29:30.242 INFO:journalctl@ceph.mon.c.smithi164.stdout: read_amp_bytes_per_bit: 0 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout: format_version: 5 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout: enable_index_compression: 1 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_align: 0 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout: max_auto_readahead_size: 262144 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout: prepopulate_block_cache: 0 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout: initial_auto_readahead_size: 8192 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compression: NoCompression 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T10:29:30.243 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T10:29:30.244 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.num_levels: 7 2024-09-16T10:29:30.244 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T10:29:30.244 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T10:29:30.244 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T10:29:30.244 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T10:29:30.244 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T10:29:30.244 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T10:29:30.244 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T10:29:30.244 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T10:29:30.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T10:29:30.246 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T10:29:30.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.inplace_update_support: 0 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T10:29:30.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.bloom_locality: 0 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.max_successive_merges: 0 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.ttl: 2592000 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.enable_blob_files: false 2024-09-16T10:29:30.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.min_blob_size: 0 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 005a96f9-3624-4fb5-bc1a-14be6be210ae 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726482570201644, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T10:29:30.250 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726482570202047, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726482570, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "005a96f9-3624-4fb5-bc1a-14be6be210ae", "db_session_id": "J9L3EBPT7Q81W82XPMSV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726482570202141, "job": 1, "event": "recovery_finished"} 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e03a884e00 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: DB pointer 0x55e03a978000 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** DB Stats ** 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T10:29:30.251 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** Compaction Stats [default] ** 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** Compaction Stats [default] ** 2024-09-16T10:29:30.252 INFO:journalctl@ceph.mon.c.smithi164.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T10:29:30.253 INFO:journalctl@ceph.mon.c.smithi164.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T10:29:30.253 INFO:journalctl@ceph.mon.c.smithi164.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T10:29:30.253 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-09-16T10:29:30.253 INFO:journalctl@ceph.mon.c.smithi164.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T10:29:30.253 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-09-16T10:29:30.253 INFO:journalctl@ceph.mon.c.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T10:29:30.253 INFO:journalctl@ceph.mon.c.smithi164.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T10:29:30.253 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T10:29:30.253 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T10:29:30.253 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout: Block cache BinnedLRUCache@0x55e03a8591f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: using public_addrv [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: starting mon.c rank -1 at public addrs [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] at bind addrs [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(???) e0 preinit fsid 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:29:30.254 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).mds e1 new map 2024-09-16T10:29:30.255 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-16T10:29:30.255 INFO:journalctl@ceph.mon.c.smithi164.stdout: e1 2024-09-16T10:29:30.255 INFO:journalctl@ceph.mon.c.smithi164.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T10:29:30.255 INFO:journalctl@ceph.mon.c.smithi164.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T10:29:30.255 INFO:journalctl@ceph.mon.c.smithi164.stdout: legacy client fscid: -1 2024-09-16T10:29:30.255 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-09-16T10:29:30.255 INFO:journalctl@ceph.mon.c.smithi164.stdout: No filesystems configured 2024-09-16T10:29:30.255 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T10:29:30.255 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T10:29:30.255 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T10:29:30.255 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T10:29:30.256 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T10:29:30.256 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T10:29:30.256 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T10:29:30.256 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T10:29:30.256 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T10:29:30.256 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T10:29:30.256 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:30.256 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T10:29:30.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:30.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:29:30.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:30.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:30.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-16T10:29:30.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/3272425435' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/3272425435' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:30.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi164:172.21.15.164=a;smithi164:[v2:172.21.15.164:3301,v1:172.21.15.164:6790]=c;smithi175:172.21.15.175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: Saving service mon spec with placement smithi164:172.21.15.164=a;smithi164:[v2:172.21.15.164:3301,v1:172.21.15.164:6790]=c;smithi175:172.21.15.175=b;count:3 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: Deploying daemon mon.b on smithi175 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/2977708483' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:30.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.a calling monitor election 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.b calling monitor election 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: monmap e2: 2 mons at {a=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: fsmap 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:30.260 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mgrmap e13: y(active, since 39s) 2024-09-16T10:29:30.261 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: overall HEALTH_OK 2024-09-16T10:29:30.261 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.261 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.261 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:30.261 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:30.261 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:30.261 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: Deploying daemon mon.c on smithi164 2024-09-16T10:29:30.261 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/1015729795' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T10:29:30.261 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:30.262 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:30.262 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 ceph-mon[40346]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T10:29:30.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 bash[40308]: 35e6e621943d6ca51c68140c0ec6923d0dd32de14dbe956cae468679efa8d471 2024-09-16T10:29:30.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:30 smithi164 systemd[1]: Started Ceph mon.c for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:29:35.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:29:35.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: mon.a calling monitor election 2024-09-16T10:29:35.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: mon.b calling monitor election 2024-09-16T10:29:35.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:35.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:35.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: monmap e3: 3 mons at {a=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: fsmap 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: mgrmap e13: y(active, since 47s) 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-16T10:29:35.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: mon.c (rank 2) addr [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] is down (out of quorum) 2024-09-16T10:29:35.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:35.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:35.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:35.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:35.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:29:35.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: mon.a calling monitor election 2024-09-16T10:29:35.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: mon.b calling monitor election 2024-09-16T10:29:35.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: monmap e3: 3 mons at {a=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: fsmap 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: mgrmap e13: y(active, since 47s) 2024-09-16T10:29:35.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-16T10:29:35.737 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-16T10:29:35.737 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-16T10:29:35.737 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: mon.c (rank 2) addr [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] is down (out of quorum) 2024-09-16T10:29:35.737 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:35.737 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:35.737 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:35.737 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:36.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:36.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:36 smithi164 ceph-mon[25828]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:36.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:36.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:36.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:36 smithi175 ceph-mon[30341]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:36.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:36.892 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-16T10:29:36.893 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":3,"fsid":"3d5087c4-7416-11ef-bceb-c7b262605968","modified":"2024-09-16T10:29:30.231862Z","created":"2024-09-16T10:27:50.856202Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3301","nonce":0},{"type":"v1","addr":"172.21.15.164:6790","nonce":0}]},"addr":"172.21.15.164:6790/0","public_addr":"172.21.15.164:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T10:29:36.893 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 3 2024-09-16T10:29:37.443 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T10:29:37.443 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T10:29:37.553 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:29:37.553 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: mon.a calling monitor election 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: mon.b calling monitor election 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: monmap e3: 3 mons at {a=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T10:29:37.554 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: fsmap 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: mgrmap e13: y(active, since 47s) 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: mon.c (rank 2) addr [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] is down (out of quorum) 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:37.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:38.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: mon.c calling monitor election 2024-09-16T10:29:38.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: mon.c calling monitor election 2024-09-16T10:29:38.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: mon.a calling monitor election 2024-09-16T10:29:38.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: mon.b calling monitor election 2024-09-16T10:29:38.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T10:29:38.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: monmap e3: 3 mons at {a=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T10:29:38.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: fsmap 2024-09-16T10:29:38.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:38.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: mgrmap e13: y(active, since 49s) 2024-09-16T10:29:38.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-16T10:29:38.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: Cluster is now healthy 2024-09-16T10:29:38.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: overall HEALTH_OK 2024-09-16T10:29:38.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:37 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:38.256 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: mon.c calling monitor election 2024-09-16T10:29:38.256 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: mon.c calling monitor election 2024-09-16T10:29:38.256 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: mon.a calling monitor election 2024-09-16T10:29:38.256 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: mon.b calling monitor election 2024-09-16T10:29:38.256 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T10:29:38.256 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: monmap e3: 3 mons at {a=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T10:29:38.256 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: fsmap 2024-09-16T10:29:38.256 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:38.256 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: mgrmap e13: y(active, since 49s) 2024-09-16T10:29:38.257 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-16T10:29:38.257 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: Cluster is now healthy 2024-09-16T10:29:38.257 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: overall HEALTH_OK 2024-09-16T10:29:38.257 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:38.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: mon.c calling monitor election 2024-09-16T10:29:38.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: mon.c calling monitor election 2024-09-16T10:29:38.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: mon.a calling monitor election 2024-09-16T10:29:38.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: mon.b calling monitor election 2024-09-16T10:29:38.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T10:29:38.257 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: monmap e3: 3 mons at {a=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],b=[v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0],c=[v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T10:29:38.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: fsmap 2024-09-16T10:29:38.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T10:29:38.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: mgrmap e13: y(active, since 49s) 2024-09-16T10:29:38.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-16T10:29:38.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: Cluster is now healthy 2024-09-16T10:29:38.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: overall HEALTH_OK 2024-09-16T10:29:38.258 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:38.300 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:29:39.152 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[40346]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:39.153 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:39.153 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:39.153 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:39.153 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:39.153 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T10:29:39.153 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:39.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[25828]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:39.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:39.154 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:39.154 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:39.154 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:39.154 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T10:29:39.154 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:38 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:39.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:38 smithi175 ceph-mon[30341]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:39.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:38 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:29:39.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:38 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:39.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:38 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:39.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:38 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:39.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:38 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T10:29:39.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:38 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:40.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:39 smithi175 ceph-mon[30341]: Reconfiguring mon.c (monmap changed)... 2024-09-16T10:29:40.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:39 smithi175 ceph-mon[30341]: Reconfiguring daemon mon.c on smithi164 2024-09-16T10:29:40.249 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:39 smithi164 ceph-mon[25828]: Reconfiguring mon.c (monmap changed)... 2024-09-16T10:29:40.249 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:39 smithi164 ceph-mon[25828]: Reconfiguring daemon mon.c on smithi164 2024-09-16T10:29:40.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:39 smithi164 ceph-mon[40346]: Reconfiguring mon.c (monmap changed)... 2024-09-16T10:29:40.249 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:39 smithi164 ceph-mon[40346]: Reconfiguring daemon mon.c on smithi164 2024-09-16T10:29:41.092 INFO:teuthology.orchestra.run.smithi164.stdout:# minimal ceph.conf for 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:29:41.092 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-09-16T10:29:41.092 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:29:41.092 INFO:teuthology.orchestra.run.smithi164.stdout: mon_host = [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] 2024-09-16T10:29:41.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[25828]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:41.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:41.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:41.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:41.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T10:29:41.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:41.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[40346]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:41.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:41.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:41.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:41.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T10:29:41.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:40 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:41.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:40 smithi175 ceph-mon[30341]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:41.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:40 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:41.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:40 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:41.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:40 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T10:29:41.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:40 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T10:29:41.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:40 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:41.686 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T10:29:41.686 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:29:41.686 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T10:29:41.713 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:29:41.714 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:29:41.778 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:29:41.778 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T10:29:41.806 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:29:41.806 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:29:41.872 INFO:tasks.cephadm:Adding mgr.y on smithi164 2024-09-16T10:29:41.872 INFO:tasks.cephadm:Adding mgr.x on smithi175 2024-09-16T10:29:41.873 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi164=y;smithi175=x' 2024-09-16T10:29:42.098 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:29:42.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[25828]: Reconfiguring mon.b (monmap changed)... 2024-09-16T10:29:42.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[25828]: Reconfiguring daemon mon.b on smithi175 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/2878676303' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[40346]: Reconfiguring mon.b (monmap changed)... 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[40346]: Reconfiguring daemon mon.b on smithi175 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/2878676303' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:42.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:42.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:42.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:42.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:42.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:41 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:42.218 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:41 smithi175 ceph-mon[30341]: Reconfiguring mon.b (monmap changed)... 2024-09-16T10:29:42.218 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:41 smithi175 ceph-mon[30341]: Reconfiguring daemon mon.b on smithi175 2024-09-16T10:29:42.218 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:41 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/2878676303' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:42.218 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:41 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:42.218 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:41 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:42.218 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:41 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:42.218 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:41 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:42.218 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:41 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:42.219 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:41 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:43.142 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:42 smithi175 ceph-mon[30341]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:43.440 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:42 smithi164 ceph-mon[25828]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:43.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:42 smithi164 ceph-mon[40346]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:43.771 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mgr update... 2024-09-16T10:29:44.370 DEBUG:teuthology.orchestra.run.smithi175:mgr.x> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T10:29:44.372 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T10:29:44.372 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:29:44.372 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T10:29:44.389 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T10:29:44.389 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/nvme4n1 2024-09-16T10:29:44.445 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/nvme4n1 2024-09-16T10:29:44.445 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T10:29:44.446 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-16T10:29:44.446 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T10:29:44.446 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T10:29:44.446 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 10:29:00.045616272 +0000 2024-09-16T10:29:44.446 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 10:27:11.927515659 +0000 2024-09-16T10:29:44.446 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 10:27:11.927515659 +0000 2024-09-16T10:29:44.446 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 10:27:11.923515729 +0000 2024-09-16T10:29:44.446 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-16T10:29:44.509 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T10:29:44.509 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T10:29:44.510 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000250094 s, 2.0 MB/s 2024-09-16T10:29:44.511 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-16T10:29:44.568 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/nvme3n1 2024-09-16T10:29:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/nvme3n1 2024-09-16T10:29:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T10:29:44.622 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 1028 Links: 1 Device type: 103,6 2024-09-16T10:29:44.622 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T10:29:44.623 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T10:29:44.623 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 10:29:00.034616465 +0000 2024-09-16T10:29:44.623 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 10:27:11.791518051 +0000 2024-09-16T10:29:44.623 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 10:27:11.791518051 +0000 2024-09-16T10:29:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 10:27:11.786518139 +0000 2024-09-16T10:29:44.623 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-16T10:29:44.686 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T10:29:44.686 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T10:29:44.686 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000240038 s, 2.1 MB/s 2024-09-16T10:29:44.687 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-16T10:29:44.744 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/nvme2n1 2024-09-16T10:29:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/nvme2n1 2024-09-16T10:29:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T10:29:44.801 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-16T10:29:44.801 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T10:29:44.801 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T10:29:44.801 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 10:29:00.023616658 +0000 2024-09-16T10:29:44.801 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 10:27:11.653520478 +0000 2024-09-16T10:29:44.801 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 10:27:11.653520478 +0000 2024-09-16T10:29:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 10:27:11.648520566 +0000 2024-09-16T10:29:44.801 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-16T10:29:44.864 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T10:29:44.864 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T10:29:44.864 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000210262 s, 2.4 MB/s 2024-09-16T10:29:44.865 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-16T10:29:44.922 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/nvme1n1 2024-09-16T10:29:44.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:44.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi164=y;smithi175=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:29:44.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: Saving service mgr spec with placement smithi164=y;smithi175=x;count:2 2024-09-16T10:29:44.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:44.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:44.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:44.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:44.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:44.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T10:29:44.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T10:29:44.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T10:29:44.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:44.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[25828]: Deploying daemon mgr.x on smithi175 2024-09-16T10:29:44.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:44.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi164=y;smithi175=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:29:44.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: Saving service mgr spec with placement smithi164=y;smithi175=x;count:2 2024-09-16T10:29:44.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:44.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:44.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:44.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:44.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:44.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T10:29:44.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T10:29:44.946 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T10:29:44.946 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:44.946 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:44 smithi164 ceph-mon[40346]: Deploying daemon mgr.x on smithi175 2024-09-16T10:29:44.960 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/nvme1n1 2024-09-16T10:29:44.960 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T10:29:44.960 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-16T10:29:44.960 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T10:29:44.960 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T10:29:44.960 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 10:29:00.012616851 +0000 2024-09-16T10:29:44.961 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 10:27:11.513522941 +0000 2024-09-16T10:29:44.961 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 10:27:11.513522941 +0000 2024-09-16T10:29:44.961 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 10:27:11.508523028 +0000 2024-09-16T10:29:44.961 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-16T10:29:44.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:44.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi164=y;smithi175=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:29:44.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: Saving service mgr spec with placement smithi164=y;smithi175=x;count:2 2024-09-16T10:29:44.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:44.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:44.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:44.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:44.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:44.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T10:29:44.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T10:29:44.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T10:29:44.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:44.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:44 smithi175 ceph-mon[30341]: Deploying daemon mgr.x on smithi175 2024-09-16T10:29:45.023 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T10:29:45.023 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T10:29:45.023 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000210112 s, 2.4 MB/s 2024-09-16T10:29:45.025 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-16T10:29:45.080 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:29:45.080 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T10:29:45.099 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-16T10:29:45.099 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/nvme4n1 2024-09-16T10:29:45.155 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/nvme4n1 2024-09-16T10:29:45.224 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T10:29:45.224 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 1039 Links: 1 Device type: 103,8 2024-09-16T10:29:45.224 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T10:29:45.224 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T10:29:45.224 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-16 10:29:11.237551829 +0000 2024-09-16T10:29:45.224 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-16 10:27:13.559293040 +0000 2024-09-16T10:29:45.224 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-16 10:27:13.559293040 +0000 2024-09-16T10:29:45.224 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-16 10:27:13.554293114 +0000 2024-09-16T10:29:45.225 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-16T10:29:45.249 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-16T10:29:45.249 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-16T10:29:45.249 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000253929 s, 2.0 MB/s 2024-09-16T10:29:45.251 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-16T10:29:45.307 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/nvme3n1 2024-09-16T10:29:45.363 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/nvme3n1 2024-09-16T10:29:45.363 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T10:29:45.363 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-16T10:29:45.363 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T10:29:45.363 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T10:29:45.363 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-16 10:29:11.226551992 +0000 2024-09-16T10:29:45.363 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-16 10:27:13.422295063 +0000 2024-09-16T10:29:45.363 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-16 10:27:13.422295063 +0000 2024-09-16T10:29:45.363 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-16 10:27:13.417295137 +0000 2024-09-16T10:29:45.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-16T10:29:45.426 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-16T10:29:45.426 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-16T10:29:45.426 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000281836 s, 1.8 MB/s 2024-09-16T10:29:45.428 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-16T10:29:45.488 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/nvme2n1 2024-09-16T10:29:45.544 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/nvme2n1 2024-09-16T10:29:45.544 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T10:29:45.544 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-16T10:29:45.545 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T10:29:45.545 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T10:29:45.545 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-16 10:29:11.216552140 +0000 2024-09-16T10:29:45.545 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-16 10:27:13.279297175 +0000 2024-09-16T10:29:45.545 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-16 10:27:13.279297175 +0000 2024-09-16T10:29:45.545 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-16 10:27:13.274297248 +0000 2024-09-16T10:29:45.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-16T10:29:45.609 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-16T10:29:45.609 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-16T10:29:45.609 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.00024233 s, 2.1 MB/s 2024-09-16T10:29:45.610 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-16T10:29:45.673 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/nvme1n1 2024-09-16T10:29:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/nvme1n1 2024-09-16T10:29:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-16T10:29:45.729 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-16T10:29:45.729 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-16T10:29:45.729 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-16T10:29:45.729 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-16 10:29:11.205552304 +0000 2024-09-16T10:29:45.729 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-16 10:27:13.138299257 +0000 2024-09-16T10:29:45.729 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-16 10:27:13.138299257 +0000 2024-09-16T10:29:45.729 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-16 10:27:13.133299331 +0000 2024-09-16T10:29:45.730 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-16T10:29:45.801 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-16T10:29:45.801 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-16T10:29:45.801 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000214048 s, 2.4 MB/s 2024-09-16T10:29:45.803 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-16T10:29:45.860 INFO:tasks.cephadm:Deploying osd.0 on smithi164 with /dev/nvme1n1... 2024-09-16T10:29:45.860 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-16T10:29:46.054 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:29:46.317 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-mgr[32427]: -- 172.21.15.175:0/419602757 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562356c9ed00 con 0x562356c73400 2024-09-16T10:29:46.573 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:46 smithi175 bash[32389]: e2b9b8042acade58c71c4dab9338d8576da94ce67f14892475db1bc798b8c51f 2024-09-16T10:29:46.573 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:46 smithi175 systemd[1]: Started Ceph mgr.x for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:29:46.573 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:46.350+0000 7f80b369e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T10:29:46.574 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:46.416+0000 7f80b369e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T10:29:46.574 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:46.571+0000 7f80b369e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T10:29:46.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[40346]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:46.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[25828]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:46.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:46.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:46.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:46.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:46.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:46.951 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:46.641+0000 7f80b369e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T10:29:46.951 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:46.809+0000 7f80b369e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:29:46.951 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:46.948+0000 7f80b369e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T10:29:46.951 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-mon[30341]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:46.951 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:46.951 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:46.951 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:46.951 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:46.951 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:46 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:47.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:47.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:47.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:47.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:47.113 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:46 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:47.485 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:47.273+0000 7f80b369e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T10:29:47.485 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:47.334+0000 7f80b369e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T10:29:47.786 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/nvme1n1 2024-09-16T10:29:47.786 INFO:teuthology.orchestra.run.smithi164.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T10:29:47.790 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-16T10:29:47.807 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-09-16T10:29:47.807 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-09-16T10:29:47.807 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0144188 s, 727 MB/s 2024-09-16T10:29:47.807 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-09-16T10:29:47.985 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:47.527+0000 7f80b369e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T10:29:48.563 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:48.563 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:48.563 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:48.564 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:48.564 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:48.564 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T10:29:48.564 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T10:29:48.564 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T10:29:48.564 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:48.564 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: Reconfiguring daemon mgr.y on smithi164 2024-09-16T10:29:48.564 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:48.564 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:48.564 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:48.564 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:48.565 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:48.565 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:48.565 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:48.565 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T10:29:48.565 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T10:29:48.565 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T10:29:48.565 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:48.565 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: Reconfiguring daemon mgr.y on smithi164 2024-09-16T10:29:48.565 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:48.565 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:48 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:48.582 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:/dev/nvme1n1 2024-09-16T10:29:48.735 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:48.396+0000 7f80b369e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T10:29:48.735 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:48.460+0000 7f80b369e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T10:29:48.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:48.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:48.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:48.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:48.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:48.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T10:29:48.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T10:29:48.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T10:29:48.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:48.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: Reconfiguring daemon mgr.y on smithi164 2024-09-16T10:29:48.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:48.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:48 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:49.004 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:29:49.424 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:49.169+0000 7f80b369e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T10:29:49.424 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:49.282+0000 7f80b369e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T10:29:49.424 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:49.421+0000 7f80b369e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:29:49.694 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:49.481+0000 7f80b369e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T10:29:49.694 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:49.625+0000 7f80b369e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T10:29:49.695 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:49.691+0000 7f80b369e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T10:29:49.972 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:49.750+0000 7f80b369e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T10:29:49.972 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:49.904+0000 7f80b369e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T10:29:49.972 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:49.969+0000 7f80b369e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T10:29:50.235 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:50.037+0000 7f80b369e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T10:29:50.235 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:50.120+0000 7f80b369e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T10:29:50.235 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:50.177+0000 7f80b369e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T10:29:50.735 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:50.235+0000 7f80b369e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T10:29:51.235 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:29:51 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:29:51.002+0000 7f80b369e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T10:29:51.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:50 smithi175 ceph-mon[30341]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:51.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:50 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:51.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:50 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:51.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:50 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:51.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:50 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:51.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:50 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:51.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:50 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:51.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[25828]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:51.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:51.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:51.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:51.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:51.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:51.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:51.284 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[40346]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:51.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:51.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:51.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:29:51.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:51.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:29:51.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:50 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:29:52.231 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[25828]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:29:52.231 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:29:52.231 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:29:52.231 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:52.231 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[25828]: Standby manager daemon x started 2024-09-16T10:29:52.231 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[25828]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T10:29:52.231 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[25828]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T10:29:52.231 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[25828]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T10:29:52.231 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[25828]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T10:29:52.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[40346]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:29:52.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:29:52.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:29:52.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:52.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[40346]: Standby manager daemon x started 2024-09-16T10:29:52.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[40346]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T10:29:52.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[40346]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T10:29:52.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[40346]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T10:29:52.232 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:51 smithi164 ceph-mon[40346]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T10:29:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:51 smithi175 ceph-mon[30341]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:29:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:51 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:29:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:51 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:29:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:51 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:29:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:51 smithi175 ceph-mon[30341]: Standby manager daemon x started 2024-09-16T10:29:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:51 smithi175 ceph-mon[30341]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T10:29:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:51 smithi175 ceph-mon[30341]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T10:29:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:51 smithi175 ceph-mon[30341]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T10:29:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:51 smithi175 ceph-mon[30341]: from='mgr.? 172.21.15.175:0/426658686' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T10:29:53.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:52 smithi164 ceph-mon[25828]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:53.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:52 smithi164 ceph-mon[25828]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-16T10:29:53.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:52 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T10:29:53.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:52 smithi164 ceph-mon[40346]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:53.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:52 smithi164 ceph-mon[40346]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-16T10:29:53.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:52 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T10:29:53.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:52 smithi175 ceph-mon[30341]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:53.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:52 smithi175 ceph-mon[30341]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-16T10:29:53.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:52 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T10:29:54.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:53 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/2390970540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a8ae783-a95a-4a6e-ad9b-fa68ce6256a8"}]: dispatch 2024-09-16T10:29:54.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:53 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/2390970540' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a8ae783-a95a-4a6e-ad9b-fa68ce6256a8"}]': finished 2024-09-16T10:29:54.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:53 smithi164 ceph-mon[25828]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T10:29:54.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:53 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:29:54.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:53 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/2390970540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a8ae783-a95a-4a6e-ad9b-fa68ce6256a8"}]: dispatch 2024-09-16T10:29:54.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:53 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/2390970540' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a8ae783-a95a-4a6e-ad9b-fa68ce6256a8"}]': finished 2024-09-16T10:29:54.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:53 smithi164 ceph-mon[40346]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T10:29:54.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:53 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:29:54.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:53 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/2390970540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a8ae783-a95a-4a6e-ad9b-fa68ce6256a8"}]: dispatch 2024-09-16T10:29:54.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:53 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/2390970540' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a8ae783-a95a-4a6e-ad9b-fa68ce6256a8"}]': finished 2024-09-16T10:29:54.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:53 smithi175 ceph-mon[30341]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T10:29:54.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:53 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:29:55.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:54 smithi175 ceph-mon[30341]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:55.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:54 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/3615973467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:29:55.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:54 smithi164 ceph-mon[40346]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:55.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:54 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/3615973467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:29:55.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:54 smithi164 ceph-mon[25828]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:55.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:54 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3615973467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:29:57.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:56 smithi164 ceph-mon[25828]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:57.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:56 smithi164 ceph-mon[40346]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:57.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:56 smithi175 ceph-mon[30341]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:59.219 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:58 smithi175 ceph-mon[30341]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:59.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:58 smithi164 ceph-mon[25828]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:29:59.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:58 smithi164 ceph-mon[40346]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:00.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T10:30:00.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:29:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:00.289 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T10:30:00.289 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:29:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:00.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T10:30:00.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:29:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:01.001 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:00 smithi164 ceph-mon[40346]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:01.001 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:00 smithi164 ceph-mon[40346]: Deploying daemon osd.0 on smithi164 2024-09-16T10:30:01.001 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:00 smithi164 ceph-mon[40346]: overall HEALTH_OK 2024-09-16T10:30:01.001 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:00 smithi164 ceph-mon[25828]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:01.002 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:00 smithi164 ceph-mon[25828]: Deploying daemon osd.0 on smithi164 2024-09-16T10:30:01.002 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:00 smithi164 ceph-mon[25828]: overall HEALTH_OK 2024-09-16T10:30:01.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:00 smithi175 ceph-mon[30341]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:01.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:00 smithi175 ceph-mon[30341]: Deploying daemon osd.0 on smithi164 2024-09-16T10:30:01.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:00 smithi175 ceph-mon[30341]: overall HEALTH_OK 2024-09-16T10:30:03.189 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:02 smithi164 ceph-mon[40346]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:03.189 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:02 smithi164 ceph-mon[25828]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:03.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:02 smithi175 ceph-mon[30341]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:05.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:04 smithi175 ceph-mon[30341]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:05.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:04 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:05.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:04 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:05.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:04 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:05.290 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:04 smithi164 ceph-mon[25828]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:05.290 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:04 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:05.290 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:04 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:05.290 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:04 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:04 smithi164 ceph-mon[40346]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:04 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:04 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:04 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:07.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:06 smithi164 ceph-mon[25828]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:07.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:06 smithi164 ceph-mon[40346]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:07.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:06 smithi175 ceph-mon[30341]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:07.866 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 0 on host 'smithi164' 2024-09-16T10:30:08.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:08.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:08.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[40346]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:08.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[40346]: from='osd.0 [v2:172.21.15.164:6802/732894384,v1:172.21.15.164:6803/732894384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T10:30:08.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[40346]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T10:30:08.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:08.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:08.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:08.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[25828]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:08.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[25828]: from='osd.0 [v2:172.21.15.164:6802/732894384,v1:172.21.15.164:6803/732894384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T10:30:08.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[25828]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T10:30:08.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:08.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:08 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:08 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:08 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:08.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:08 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:08.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:08 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:08 smithi175 ceph-mon[30341]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:08.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:08 smithi175 ceph-mon[30341]: from='osd.0 [v2:172.21.15.164:6802/732894384,v1:172.21.15.164:6803/732894384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T10:30:08.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:08 smithi175 ceph-mon[30341]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T10:30:08.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:08 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:08.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:08 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:08 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:08.611 DEBUG:teuthology.orchestra.run.smithi164:osd.0> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T10:30:08.613 INFO:tasks.cephadm:Deploying osd.1 on smithi164 with /dev/nvme2n1... 2024-09-16T10:30:08.613 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-16T10:30:09.385 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:30:09.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-mon[25828]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T10:30:09.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-mon[25828]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T10:30:09.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:09.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-mon[25828]: from='osd.0 [v2:172.21.15.164:6802/732894384,v1:172.21.15.164:6803/732894384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:09.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-mon[25828]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:09.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-mon[40346]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T10:30:09.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-mon[40346]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T10:30:09.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:09.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-mon[40346]: from='osd.0 [v2:172.21.15.164:6802/732894384,v1:172.21.15.164:6803/732894384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:09.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-mon[40346]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:09.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:09 smithi175 ceph-mon[30341]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T10:30:09.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:09 smithi175 ceph-mon[30341]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T10:30:09.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:09 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:09.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:09 smithi175 ceph-mon[30341]: from='osd.0 [v2:172.21.15.164:6802/732894384,v1:172.21.15.164:6803/732894384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:09.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:09 smithi175 ceph-mon[30341]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:10.149 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0[44606]: 2024-09-16T10:30:09.825+0000 7f7996e32640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T10:30:10.150 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:30:09 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0[44606]: 2024-09-16T10:30:09.828+0000 7f798e43c640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T10:30:10.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[25828]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:30:10.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[25828]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T10:30:10.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:10.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:10.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[25828]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:10.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[25828]: from='osd.0 ' entity='osd.0' 2024-09-16T10:30:10.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[40346]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:30:10.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[40346]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T10:30:10.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:10.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:10.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[40346]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:10.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:10 smithi164 ceph-mon[40346]: from='osd.0 ' entity='osd.0' 2024-09-16T10:30:10.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:10 smithi175 ceph-mon[30341]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:30:10.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:10 smithi175 ceph-mon[30341]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T10:30:10.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:10 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:10.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:10 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:10.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:10 smithi175 ceph-mon[30341]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T10:30:10.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:10 smithi175 ceph-mon[30341]: from='osd.0 ' entity='osd.0' 2024-09-16T10:30:11.649 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[25828]: purged_snaps scrub starts 2024-09-16T10:30:11.650 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[25828]: purged_snaps scrub ok 2024-09-16T10:30:11.650 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:11.650 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[25828]: osd.0 [v2:172.21.15.164:6802/732894384,v1:172.21.15.164:6803/732894384] boot 2024-09-16T10:30:11.650 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[25828]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T10:30:11.650 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:11.650 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[40346]: purged_snaps scrub starts 2024-09-16T10:30:11.650 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[40346]: purged_snaps scrub ok 2024-09-16T10:30:11.650 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:11.651 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[40346]: osd.0 [v2:172.21.15.164:6802/732894384,v1:172.21.15.164:6803/732894384] boot 2024-09-16T10:30:11.651 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[40346]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T10:30:11.651 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:11 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:11.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:11 smithi175 ceph-mon[30341]: purged_snaps scrub starts 2024-09-16T10:30:11.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:11 smithi175 ceph-mon[30341]: purged_snaps scrub ok 2024-09-16T10:30:11.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:11 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:11.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:11 smithi175 ceph-mon[30341]: osd.0 [v2:172.21.15.164:6802/732894384,v1:172.21.15.164:6803/732894384] boot 2024-09-16T10:30:11.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:11 smithi175 ceph-mon[30341]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T10:30:11.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:11 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:30:11.911 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/nvme2n1 2024-09-16T10:30:11.911 INFO:teuthology.orchestra.run.smithi164.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T10:30:11.915 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-16T10:30:11.931 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-09-16T10:30:11.931 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-09-16T10:30:11.931 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0135593 s, 773 MB/s 2024-09-16T10:30:11.931 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-09-16T10:30:12.617 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:/dev/nvme2n1 2024-09-16T10:30:12.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:12 smithi164 ceph-mon[25828]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:12.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:12 smithi164 ceph-mon[25828]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T10:30:12.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:12 smithi164 ceph-mon[40346]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:12.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:12 smithi164 ceph-mon[40346]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T10:30:13.222 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:30:13.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:12 smithi175 ceph-mon[30341]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:13.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:12 smithi175 ceph-mon[30341]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T10:30:14.371 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[40346]: Detected new or changed devices on smithi164 2024-09-16T10:30:14.371 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.371 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.371 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:30:14.371 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[40346]: Adjusting osd_memory_target on smithi164 to 16064M 2024-09-16T10:30:14.371 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.371 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:14.372 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:14.372 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.372 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[40346]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:14.372 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[25828]: Detected new or changed devices on smithi164 2024-09-16T10:30:14.372 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.372 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.372 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:30:14.372 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[25828]: Adjusting osd_memory_target on smithi164 to 16064M 2024-09-16T10:30:14.372 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.372 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:14.372 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:14.373 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.373 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:14 smithi164 ceph-mon[25828]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:14.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:14 smithi175 ceph-mon[30341]: Detected new or changed devices on smithi164 2024-09-16T10:30:14.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:30:14.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:14 smithi175 ceph-mon[30341]: Adjusting osd_memory_target on smithi164 to 16064M 2024-09-16T10:30:14.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:14.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:14.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:14.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:14 smithi175 ceph-mon[30341]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:15.567 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:15 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:30:15.568 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:15 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:30:15.568 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:15 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:15.568 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:15 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:30:15.568 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:15 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:30:15.568 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:15 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:15.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:15 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:30:15.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:15 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:30:15.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:15 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:16.430 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:16 smithi164 ceph-mon[25828]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:30:16.430 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:16 smithi164 ceph-mon[25828]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:16.430 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:16 smithi164 ceph-mon[40346]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:30:16.430 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:16 smithi164 ceph-mon[40346]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:16.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:16 smithi175 ceph-mon[30341]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:30:16.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:16 smithi175 ceph-mon[30341]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:17.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:17 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3850607351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da925d4-94e1-4f33-9c3f-90d6ea8574ba"}]: dispatch 2024-09-16T10:30:17.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:17 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da925d4-94e1-4f33-9c3f-90d6ea8574ba"}]: dispatch 2024-09-16T10:30:17.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:17 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da925d4-94e1-4f33-9c3f-90d6ea8574ba"}]': finished 2024-09-16T10:30:17.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:17 smithi164 ceph-mon[25828]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T10:30:17.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:17 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:17.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:17 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/3850607351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da925d4-94e1-4f33-9c3f-90d6ea8574ba"}]: dispatch 2024-09-16T10:30:17.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:17 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da925d4-94e1-4f33-9c3f-90d6ea8574ba"}]: dispatch 2024-09-16T10:30:17.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:17 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da925d4-94e1-4f33-9c3f-90d6ea8574ba"}]': finished 2024-09-16T10:30:17.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:17 smithi164 ceph-mon[40346]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T10:30:17.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:17 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:17.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:17 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/3850607351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da925d4-94e1-4f33-9c3f-90d6ea8574ba"}]: dispatch 2024-09-16T10:30:17.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:17 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da925d4-94e1-4f33-9c3f-90d6ea8574ba"}]: dispatch 2024-09-16T10:30:17.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:17 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da925d4-94e1-4f33-9c3f-90d6ea8574ba"}]': finished 2024-09-16T10:30:17.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:17 smithi175 ceph-mon[30341]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T10:30:17.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:17 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:18.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:18 smithi164 ceph-mon[25828]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:18.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:18 smithi164 ceph-mon[40346]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:18.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:18 smithi175 ceph-mon[30341]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:19.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:19 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1223699598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:30:19.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:19 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1223699598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:30:19.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:19 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1223699598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:30:20.547 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:20 smithi164 ceph-mon[40346]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:20.548 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:20 smithi164 ceph-mon[25828]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:20.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:20 smithi175 ceph-mon[30341]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:22.874 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:22 smithi164 ceph-mon[25828]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:22.874 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:22 smithi164 ceph-mon[40346]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:23.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:22 smithi175 ceph-mon[30341]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:24.107 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:23 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T10:30:24.107 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:23 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:24.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:23 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T10:30:24.107 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:23 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:24.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:23 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T10:30:24.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:23 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:25.038 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:24 smithi164 ceph-mon[25828]: Deploying daemon osd.1 on smithi164 2024-09-16T10:30:25.039 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:24 smithi164 ceph-mon[25828]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:25.039 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:24 smithi164 ceph-mon[40346]: Deploying daemon osd.1 on smithi164 2024-09-16T10:30:25.039 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:24 smithi164 ceph-mon[40346]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:25.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:24 smithi175 ceph-mon[30341]: Deploying daemon osd.1 on smithi164 2024-09-16T10:30:25.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:24 smithi175 ceph-mon[30341]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:26.997 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:26 smithi164 ceph-mon[25828]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:26.998 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:26 smithi164 ceph-mon[40346]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:27.037 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:26 smithi175 ceph-mon[30341]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:29.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:28 smithi164 ceph-mon[40346]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:29.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:28 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:29.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:28 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:29.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:28 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:29.052 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:28 smithi164 ceph-mon[25828]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:29.053 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:28 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:29.053 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:28 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:29.053 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:28 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:29.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:28 smithi175 ceph-mon[30341]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:29.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:28 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:29.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:28 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:29.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:28 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:30.820 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:30 smithi164 ceph-mon[25828]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:30.820 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:30 smithi164 ceph-mon[40346]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:31.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:30 smithi175 ceph-mon[30341]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:31.948 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 1 on host 'smithi164' 2024-09-16T10:30:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[40346]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T10:30:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[40346]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:32.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:32.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[25828]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T10:30:32.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[25828]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:32.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:32.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:32.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:32.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:32 smithi175 ceph-mon[30341]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T10:30:32.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:32 smithi175 ceph-mon[30341]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:32.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:32.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:32.566 DEBUG:teuthology.orchestra.run.smithi164:osd.1> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T10:30:32.569 INFO:tasks.cephadm:Deploying osd.2 on smithi164 with /dev/nvme3n1... 2024-09-16T10:30:32.569 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-16T10:30:33.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:33 smithi164 ceph-mon[25828]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T10:30:33.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:33 smithi164 ceph-mon[25828]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T10:30:33.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:33 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:33.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:33 smithi164 ceph-mon[25828]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:33.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:33 smithi164 ceph-mon[40346]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T10:30:33.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:33 smithi164 ceph-mon[40346]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T10:30:33.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:33 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:33.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:33 smithi164 ceph-mon[40346]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:33.195 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:30:33 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1[52041]: 2024-09-16T10:30:33.078+0000 7f7792af2640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T10:30:33.195 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:30:33 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1[52041]: 2024-09-16T10:30:33.081+0000 7f778e91b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T10:30:33.269 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:30:33.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:33 smithi175 ceph-mon[30341]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T10:30:33.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:33 smithi175 ceph-mon[30341]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T10:30:33.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:33 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:33.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:33 smithi175 ceph-mon[30341]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:34.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:34 smithi164 ceph-mon[25828]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:30:34.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:34 smithi164 ceph-mon[25828]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T10:30:34.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:34 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:34.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:34 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:34.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:34 smithi164 ceph-mon[25828]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:34.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:34 smithi164 ceph-mon[40346]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:30:34.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:34 smithi164 ceph-mon[40346]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T10:30:34.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:34 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:34.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:34 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:34.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:34 smithi164 ceph-mon[40346]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:34.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:34 smithi175 ceph-mon[30341]: from='osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:30:34.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:34 smithi175 ceph-mon[30341]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T10:30:34.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:34 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:34.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:34 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:34.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:34 smithi175 ceph-mon[30341]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T10:30:35.342 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[40346]: purged_snaps scrub starts 2024-09-16T10:30:35.342 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[40346]: purged_snaps scrub ok 2024-09-16T10:30:35.343 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:35.343 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[40346]: osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442] boot 2024-09-16T10:30:35.343 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[40346]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T10:30:35.343 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:35.343 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[25828]: purged_snaps scrub starts 2024-09-16T10:30:35.343 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[25828]: purged_snaps scrub ok 2024-09-16T10:30:35.343 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:35.344 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[25828]: osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442] boot 2024-09-16T10:30:35.344 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[25828]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T10:30:35.344 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:35 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:35.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:35 smithi175 ceph-mon[30341]: purged_snaps scrub starts 2024-09-16T10:30:35.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:35 smithi175 ceph-mon[30341]: purged_snaps scrub ok 2024-09-16T10:30:35.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:35.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:35 smithi175 ceph-mon[30341]: osd.1 [v2:172.21.15.164:6810/4121909442,v1:172.21.15.164:6811/4121909442] boot 2024-09-16T10:30:35.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:35 smithi175 ceph-mon[30341]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T10:30:35.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:35 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:30:35.816 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/nvme3n1 2024-09-16T10:30:35.816 INFO:teuthology.orchestra.run.smithi164.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T10:30:35.821 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-16T10:30:35.836 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-09-16T10:30:35.836 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-09-16T10:30:35.836 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132893 s, 789 MB/s 2024-09-16T10:30:35.837 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-09-16T10:30:36.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:36 smithi164 ceph-mon[40346]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T10:30:36.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:36 smithi164 ceph-mon[40346]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:36.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:36 smithi164 ceph-mon[25828]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T10:30:36.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:36 smithi164 ceph-mon[25828]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:36.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:36 smithi175 ceph-mon[30341]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T10:30:36.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:36 smithi175 ceph-mon[30341]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:36.790 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:/dev/nvme3n1 2024-09-16T10:30:36.982 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:30:37.437 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[40346]: Detected new or changed devices on smithi164 2024-09-16T10:30:37.437 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:37.437 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[40346]: Adjusting osd_memory_target on smithi164 to 8032M 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[25828]: Detected new or changed devices on smithi164 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:30:37.438 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:30:37.439 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[25828]: Adjusting osd_memory_target on smithi164 to 8032M 2024-09-16T10:30:37.439 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:37.439 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:37.439 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:37.439 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:37 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:37.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:37 smithi175 ceph-mon[30341]: Detected new or changed devices on smithi164 2024-09-16T10:30:37.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:37 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:37.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:37 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:37.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:37 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:30:37.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:37 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:30:37.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:37 smithi175 ceph-mon[30341]: Adjusting osd_memory_target on smithi164 to 8032M 2024-09-16T10:30:37.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:37 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:37.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:37 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:37.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:37 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:30:37.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:37 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:38.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:38 smithi164 ceph-mon[40346]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:38.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:38 smithi164 ceph-mon[25828]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:38.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:38 smithi175 ceph-mon[30341]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:39.346 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:39 smithi164 ceph-mon[40346]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:30:39.346 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:39 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:30:39.346 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:39 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:30:39.346 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:39 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:39.347 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:39 smithi164 ceph-mon[25828]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:30:39.347 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:39 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:30:39.347 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:39 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:30:39.347 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:39 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:39.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:39 smithi175 ceph-mon[30341]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:30:39.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:39 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:30:39.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:39 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:30:39.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:39 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:40.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:40 smithi164 ceph-mon[40346]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:40.260 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:40 smithi164 ceph-mon[25828]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:40.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:40 smithi175 ceph-mon[30341]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:41.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:41 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/2061081952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8c3465b-81e1-4ef4-8574-049b2b8e7455"}]: dispatch 2024-09-16T10:30:41.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:41 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8c3465b-81e1-4ef4-8574-049b2b8e7455"}]: dispatch 2024-09-16T10:30:41.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:41 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8c3465b-81e1-4ef4-8574-049b2b8e7455"}]': finished 2024-09-16T10:30:41.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:41 smithi164 ceph-mon[40346]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T10:30:41.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:41 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:41.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:41 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/2061081952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8c3465b-81e1-4ef4-8574-049b2b8e7455"}]: dispatch 2024-09-16T10:30:41.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:41 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8c3465b-81e1-4ef4-8574-049b2b8e7455"}]: dispatch 2024-09-16T10:30:41.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:41 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8c3465b-81e1-4ef4-8574-049b2b8e7455"}]': finished 2024-09-16T10:30:41.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:41 smithi164 ceph-mon[25828]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T10:30:41.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:41 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:41.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:41 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/2061081952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8c3465b-81e1-4ef4-8574-049b2b8e7455"}]: dispatch 2024-09-16T10:30:41.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:41 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8c3465b-81e1-4ef4-8574-049b2b8e7455"}]: dispatch 2024-09-16T10:30:41.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:41 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8c3465b-81e1-4ef4-8574-049b2b8e7455"}]': finished 2024-09-16T10:30:41.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:41 smithi175 ceph-mon[30341]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T10:30:41.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:41 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:42.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:42 smithi164 ceph-mon[25828]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:42.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:42 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/2859873210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:30:42.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:42 smithi164 ceph-mon[40346]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:42.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:42 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/2859873210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:30:42.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:42 smithi175 ceph-mon[30341]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:42.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:42 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/2859873210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:30:45.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:44 smithi164 ceph-mon[25828]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:45.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:44 smithi164 ceph-mon[40346]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:45.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:44 smithi175 ceph-mon[30341]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:47.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:46 smithi164 ceph-mon[25828]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:47.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:46 smithi164 ceph-mon[40346]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:47.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:46 smithi175 ceph-mon[30341]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:47.913 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T10:30:47.913 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:47.913 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:47 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T10:30:47.913 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:47 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:48.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:47 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T10:30:48.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:47 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:30:48.886 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:48 smithi164 ceph-mon[40346]: Deploying daemon osd.2 on smithi164 2024-09-16T10:30:48.886 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:48 smithi164 ceph-mon[40346]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:48.887 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:48 smithi164 ceph-mon[25828]: Deploying daemon osd.2 on smithi164 2024-09-16T10:30:48.887 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:48 smithi164 ceph-mon[25828]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:49.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:48 smithi175 ceph-mon[30341]: Deploying daemon osd.2 on smithi164 2024-09-16T10:30:49.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:48 smithi175 ceph-mon[30341]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:51.039 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:50 smithi164 ceph-mon[25828]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:51.039 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:50 smithi164 ceph-mon[40346]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:51.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:50 smithi175 ceph-mon[30341]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:52.786 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:52 smithi164 ceph-mon[25828]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:52.786 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:52 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:52.786 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:52 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:52.786 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:52 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:53.038 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:52 smithi164 ceph-mon[40346]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:53.038 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:52 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:53.038 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:52 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:53.038 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:52 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:53.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:52 smithi175 ceph-mon[30341]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:53.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:52 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:30:53.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:52 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:53.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:52 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:55.109 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:54 smithi164 ceph-mon[40346]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:55.109 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:54 smithi164 ceph-mon[25828]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:55.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:54 smithi175 ceph-mon[30341]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:55.782 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 2 on host 'smithi164' 2024-09-16T10:30:56.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:55 smithi164 ceph-mon[40346]: from='osd.2 [v2:172.21.15.164:6818/254163173,v1:172.21.15.164:6819/254163173]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T10:30:56.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:55 smithi164 ceph-mon[40346]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T10:30:56.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:55 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:56.081 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:55 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:56.081 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:55 smithi164 ceph-mon[25828]: from='osd.2 [v2:172.21.15.164:6818/254163173,v1:172.21.15.164:6819/254163173]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T10:30:56.081 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:55 smithi164 ceph-mon[25828]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T10:30:56.081 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:55 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:56.081 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:55 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:56.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:55 smithi175 ceph-mon[30341]: from='osd.2 [v2:172.21.15.164:6818/254163173,v1:172.21.15.164:6819/254163173]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T10:30:56.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:55 smithi175 ceph-mon[30341]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T10:30:56.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:55 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:55 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:56.402 DEBUG:teuthology.orchestra.run.smithi164:osd.2> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T10:30:56.405 INFO:tasks.cephadm:Deploying osd.3 on smithi164 with /dev/nvme4n1... 2024-09-16T10:30:56.405 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-16T10:30:56.643 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:30:57.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[40346]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:57.042 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[40346]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T10:30:57.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[40346]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T10:30:57.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:57.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[40346]: from='osd.2 [v2:172.21.15.164:6818/254163173,v1:172.21.15.164:6819/254163173]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:57.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[40346]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:57.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:57.043 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:57.043 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[25828]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:57.043 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[25828]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T10:30:57.044 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[25828]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T10:30:57.044 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:57.044 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[25828]: from='osd.2 [v2:172.21.15.164:6818/254163173,v1:172.21.15.164:6819/254163173]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:57.044 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[25828]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:57.044 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:57.044 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:57.044 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2[59757]: 2024-09-16T10:30:56.812+0000 7f2ec8a3f640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T10:30:57.044 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:30:56 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2[59757]: 2024-09-16T10:30:56.815+0000 7f2ec4868640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T10:30:57.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:56 smithi175 ceph-mon[30341]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:57.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:56 smithi175 ceph-mon[30341]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T10:30:57.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:56 smithi175 ceph-mon[30341]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T10:30:57.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:56 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:57.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:56 smithi175 ceph-mon[30341]: from='osd.2 [v2:172.21.15.164:6818/254163173,v1:172.21.15.164:6819/254163173]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:57.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:56 smithi175 ceph-mon[30341]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:30:57.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:56 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:57.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:56 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:30:58.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:57 smithi164 ceph-mon[40346]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:30:58.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:57 smithi164 ceph-mon[40346]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T10:30:58.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:57 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:58.140 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:57 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:58.140 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:57 smithi164 ceph-mon[25828]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:30:58.140 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:57 smithi164 ceph-mon[25828]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T10:30:58.140 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:57 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:58.140 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:57 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:57 smithi175 ceph-mon[30341]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:30:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:57 smithi175 ceph-mon[30341]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T10:30:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:57 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:57 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:58.987 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/nvme4n1 2024-09-16T10:30:58.987 INFO:teuthology.orchestra.run.smithi164.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T10:30:58.991 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-16T10:30:59.007 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-09-16T10:30:59.007 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-09-16T10:30:59.007 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0136217 s, 770 MB/s 2024-09-16T10:30:59.007 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[40346]: purged_snaps scrub starts 2024-09-16T10:30:59.007 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[40346]: purged_snaps scrub ok 2024-09-16T10:30:59.007 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[40346]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:59.008 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:59.008 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[40346]: osd.2 [v2:172.21.15.164:6818/254163173,v1:172.21.15.164:6819/254163173] boot 2024-09-16T10:30:59.008 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[40346]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T10:30:59.008 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:59.008 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[25828]: purged_snaps scrub starts 2024-09-16T10:30:59.008 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[25828]: purged_snaps scrub ok 2024-09-16T10:30:59.008 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[25828]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:59.008 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:59.008 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[25828]: osd.2 [v2:172.21.15.164:6818/254163173,v1:172.21.15.164:6819/254163173] boot 2024-09-16T10:30:59.008 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[25828]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T10:30:59.008 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:30:58 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:59.009 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-09-16T10:30:59.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:58 smithi175 ceph-mon[30341]: purged_snaps scrub starts 2024-09-16T10:30:59.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:58 smithi175 ceph-mon[30341]: purged_snaps scrub ok 2024-09-16T10:30:59.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:58 smithi175 ceph-mon[30341]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T10:30:59.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:58 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:59.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:58 smithi175 ceph-mon[30341]: osd.2 [v2:172.21.15.164:6818/254163173,v1:172.21.15.164:6819/254163173] boot 2024-09-16T10:30:59.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:58 smithi175 ceph-mon[30341]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T10:30:59.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:30:58 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:30:59.822 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:/dev/nvme4n1 2024-09-16T10:31:00.079 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:00 smithi164 ceph-mon[40346]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T10:31:00.079 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:00 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T10:31:00.079 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:00 smithi164 ceph-mon[25828]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T10:31:00.079 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:00 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T10:31:00.295 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:31:00.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:00 smithi175 ceph-mon[30341]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T10:31:00.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:00 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T10:31:01.411 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:01 smithi164 ceph-mon[40346]: pgmap v75: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:01.411 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:01 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T10:31:01.411 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:01 smithi164 ceph-mon[40346]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T10:31:01.411 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:01 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T10:31:01.411 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:01 smithi164 ceph-mon[25828]: pgmap v75: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:01.411 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:01 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T10:31:01.411 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:01 smithi164 ceph-mon[25828]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T10:31:01.411 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:01 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T10:31:01.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:01 smithi175 ceph-mon[30341]: pgmap v75: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:01.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:01 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T10:31:01.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:01 smithi175 ceph-mon[30341]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T10:31:01.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:01 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T10:31:02.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:02 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T10:31:02.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:02 smithi164 ceph-mon[40346]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T10:31:02.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:02 smithi164 ceph-mon[40346]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:02.444 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66336]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-16T10:31:02.444 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66336]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:02.444 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66336]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:02.444 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66336]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:02.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:02 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T10:31:02.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:02 smithi164 ceph-mon[25828]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T10:31:02.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:02 smithi164 ceph-mon[25828]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:02.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:02 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T10:31:02.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:02 smithi175 ceph-mon[30341]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T10:31:02.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:02 smithi175 ceph-mon[30341]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:02.741 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66418]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-16T10:31:02.741 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66418]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:02.741 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66418]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:02.741 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66418]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:03.003 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66525]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2024-09-16T10:31:03.004 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66525]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:03.004 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66525]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:03.004 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:31:02 smithi164 sudo[66525]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:03.272 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66565]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme2n1 2024-09-16T10:31:03.273 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66565]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:03.273 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66565]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:03.273 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66565]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:03.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T10:31:03.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:31:03.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:31:03.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:31:03.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:03.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: Detected new or changed devices on smithi164 2024-09-16T10:31:03.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:03.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:03.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:03.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: Adjusting osd_memory_target on smithi164 to 5354M 2024-09-16T10:31:03.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:03.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:03.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:03.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:03.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:03.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.275 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T10:31:03.275 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:31:03.275 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:31:03.275 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:31:03.275 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:03.275 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: Detected new or changed devices on smithi164 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: Adjusting osd_memory_target on smithi164 to 5354M 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:03.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:03.277 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:03.277 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.277 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: Detected new or changed devices on smithi164 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:03.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: Adjusting osd_memory_target on smithi164 to 5354M 2024-09-16T10:31:03.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:03.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:03.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:03.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:03.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:03.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:03.530 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66574]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme3n1 2024-09-16T10:31:03.530 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66574]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:03.530 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66574]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:03.530 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66574]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:03.800 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66578]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme3n1 2024-09-16T10:31:03.800 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66578]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:03.800 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66578]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:03.801 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66578]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:04.061 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66602]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T10:31:04.061 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66602]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:04.062 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66602]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:04.062 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:03 smithi164 sudo[66602]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:04.413 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 sudo[66626]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T10:31:04.413 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 sudo[66626]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:04.413 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 sudo[66626]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:04.413 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 sudo[66626]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:04.584 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 sudo[33079]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T10:31:04.584 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 sudo[33079]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:04.584 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 sudo[33079]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:04.584 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 sudo[33079]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:04.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:04.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[25828]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T10:31:04.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[25828]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:04.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[25828]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T10:31:04.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:31:04.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:31:04.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:31:04.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:04.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:31:04.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:31:04.694 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:31:04.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:04.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[40346]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T10:31:04.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[40346]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:04.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[40346]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T10:31:04.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:31:04.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:31:04.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:31:04.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:04.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:31:04.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:31:04.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:31:04.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 sudo[33082]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T10:31:04.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 sudo[33082]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:04.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 sudo[33082]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:04.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 sudo[33082]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:04.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:04.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 ceph-mon[30341]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T10:31:04.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 ceph-mon[30341]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:04.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 ceph-mon[30341]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T10:31:04.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:31:04.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:31:04.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:31:04.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:04.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:31:04.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:31:04.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:04 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:31:05.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 sudo[66684]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T10:31:05.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 sudo[66684]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:05.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 sudo[66684]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:05.118 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:04 smithi164 sudo[66684]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:05.395 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 sudo[66687]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T10:31:05.396 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 sudo[66687]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T10:31:05.396 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 sudo[66687]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T10:31:05.396 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 sudo[66687]: pam_unix(sudo:session): session closed for user root 2024-09-16T10:31:05.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T10:31:05.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T10:31:05.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T10:31:05.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:05.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T10:31:05.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:31:05.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:31:05.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:31:05.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/3940283504' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0662cc54-7dac-48dd-8802-8f82c6424867"}]: dispatch 2024-09-16T10:31:05.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T10:31:05.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/3940283504' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0662cc54-7dac-48dd-8802-8f82c6424867"}]': finished 2024-09-16T10:31:05.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T10:31:05.695 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:05.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T10:31:05.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T10:31:05.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T10:31:05.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:05.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T10:31:05.695 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:31:05.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:31:05.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:31:05.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3940283504' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0662cc54-7dac-48dd-8802-8f82c6424867"}]: dispatch 2024-09-16T10:31:05.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T10:31:05.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3940283504' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0662cc54-7dac-48dd-8802-8f82c6424867"}]': finished 2024-09-16T10:31:05.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T10:31:05.696 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:05 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:05.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T10:31:05.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T10:31:05.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T10:31:05.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:05.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T10:31:05.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:31:05.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:31:05.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:31:05.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/3940283504' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0662cc54-7dac-48dd-8802-8f82c6424867"}]: dispatch 2024-09-16T10:31:05.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T10:31:05.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/3940283504' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0662cc54-7dac-48dd-8802-8f82c6424867"}]': finished 2024-09-16T10:31:05.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T10:31:05.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:05 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:06.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:06 smithi164 ceph-mon[25828]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:06.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:06 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/509129168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:31:06.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:06 smithi164 ceph-mon[40346]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:06.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:06 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/509129168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:31:06.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:06 smithi175 ceph-mon[30341]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:06.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:06 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/509129168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:31:09.180 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:08 smithi164 ceph-mon[40346]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:09.180 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:08 smithi164 ceph-mon[25828]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:09.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:08 smithi175 ceph-mon[30341]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:10.924 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:10 smithi164 ceph-mon[40346]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:10.924 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:10 smithi164 ceph-mon[25828]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:11.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:10 smithi175 ceph-mon[30341]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:13.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:12 smithi164 ceph-mon[25828]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 40 KiB/s wr, 3 op/s 2024-09-16T10:31:13.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T10:31:13.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:13.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:12 smithi164 ceph-mon[25828]: Deploying daemon osd.3 on smithi164 2024-09-16T10:31:13.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:12 smithi164 ceph-mon[40346]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 40 KiB/s wr, 3 op/s 2024-09-16T10:31:13.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T10:31:13.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:13.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:12 smithi164 ceph-mon[40346]: Deploying daemon osd.3 on smithi164 2024-09-16T10:31:13.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:12 smithi175 ceph-mon[30341]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 40 KiB/s wr, 3 op/s 2024-09-16T10:31:13.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T10:31:13.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:13.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:12 smithi175 ceph-mon[30341]: Deploying daemon osd.3 on smithi164 2024-09-16T10:31:15.180 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:14 smithi164 ceph-mon[40346]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 30 KiB/s wr, 2 op/s 2024-09-16T10:31:15.181 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:14 smithi164 ceph-mon[25828]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 30 KiB/s wr, 2 op/s 2024-09-16T10:31:15.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:14 smithi175 ceph-mon[30341]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 30 KiB/s wr, 2 op/s 2024-09-16T10:31:17.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:16 smithi164 ceph-mon[25828]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 869 B/s rd, 25 KiB/s wr, 2 op/s 2024-09-16T10:31:17.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:16 smithi164 ceph-mon[40346]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 869 B/s rd, 25 KiB/s wr, 2 op/s 2024-09-16T10:31:17.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:16 smithi175 ceph-mon[30341]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 869 B/s rd, 25 KiB/s wr, 2 op/s 2024-09-16T10:31:17.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:17 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:17.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:17 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:17.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:17 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:17.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:17 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:17.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:17 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:17.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:17 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:18.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:17 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:18.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:17 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:18.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:17 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:19.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:18 smithi164 ceph-mon[40346]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T10:31:19.117 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:18 smithi164 ceph-mon[25828]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T10:31:19.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:18 smithi175 ceph-mon[30341]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T10:31:20.117 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 3 on host 'smithi164' 2024-09-16T10:31:20.883 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:20 smithi164 ceph-mon[40346]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T10:31:20.883 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:20 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:20.883 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:20 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:20.883 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:20 smithi164 ceph-mon[40346]: from='osd.3 [v2:172.21.15.164:6826/3793297178,v1:172.21.15.164:6827/3793297178]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T10:31:20.884 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:20 smithi164 ceph-mon[40346]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T10:31:20.884 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:20 smithi164 ceph-mon[25828]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T10:31:20.884 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:20 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:20.884 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:20 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:20.884 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:20 smithi164 ceph-mon[25828]: from='osd.3 [v2:172.21.15.164:6826/3793297178,v1:172.21.15.164:6827/3793297178]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T10:31:20.884 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:20 smithi164 ceph-mon[25828]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T10:31:20.909 DEBUG:teuthology.orchestra.run.smithi164:osd.3> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T10:31:20.911 INFO:tasks.cephadm:Deploying osd.4 on smithi175 with /dev/nvme1n1... 2024-09-16T10:31:20.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-16T10:31:21.114 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:31:21.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:20 smithi175 ceph-mon[30341]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T10:31:21.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:20 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:21.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:20 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:21.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:20 smithi175 ceph-mon[30341]: from='osd.3 [v2:172.21.15.164:6826/3793297178,v1:172.21.15.164:6827/3793297178]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T10:31:21.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:20 smithi175 ceph-mon[30341]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T10:31:22.207 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:22 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:22.207 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:22 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:22.207 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:22 smithi175 ceph-mon[30341]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T10:31:22.207 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:22 smithi175 ceph-mon[30341]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T10:31:22.207 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:22 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:22.207 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:22 smithi175 ceph-mon[30341]: from='osd.3 [v2:172.21.15.164:6826/3793297178,v1:172.21.15.164:6827/3793297178]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:31:22.207 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:22 smithi175 ceph-mon[30341]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:31:22.207 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:22 smithi175 ceph-mon[30341]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:22.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:22.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:22.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[40346]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T10:31:22.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[40346]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T10:31:22.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:22.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[40346]: from='osd.3 [v2:172.21.15.164:6826/3793297178,v1:172.21.15.164:6827/3793297178]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:31:22.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[40346]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:31:22.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[40346]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:22.331 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:22.331 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:22.331 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[25828]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T10:31:22.332 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[25828]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T10:31:22.332 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:22.332 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[25828]: from='osd.3 [v2:172.21.15.164:6826/3793297178,v1:172.21.15.164:6827/3793297178]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:31:22.332 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[25828]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-16T10:31:22.332 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-mon[25828]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T10:31:22.332 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3[68539]: 2024-09-16T10:31:22.131+0000 7fc2309dc640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T10:31:22.332 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:31:22 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3[68539]: 2024-09-16T10:31:22.134+0000 7fc22c805640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T10:31:22.752 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/nvme1n1 2024-09-16T10:31:22.752 INFO:teuthology.orchestra.run.smithi175.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T10:31:22.756 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-16T10:31:22.773 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-09-16T10:31:22.773 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-09-16T10:31:22.773 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0140575 s, 746 MB/s 2024-09-16T10:31:22.774 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-09-16T10:31:23.346 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:23 smithi175 ceph-mon[30341]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:31:23.346 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:23 smithi175 ceph-mon[30341]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T10:31:23.346 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:23 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:23.346 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:23 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:23.367 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi175:/dev/nvme1n1 2024-09-16T10:31:23.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:23 smithi164 ceph-mon[40346]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:31:23.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:23 smithi164 ceph-mon[40346]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T10:31:23.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:23 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:23.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:23 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:23.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:23 smithi164 ceph-mon[25828]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-16T10:31:23.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:23 smithi164 ceph-mon[25828]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T10:31:23.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:23 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:23.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:23 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:23.557 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:31:24.399 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[40346]: purged_snaps scrub starts 2024-09-16T10:31:24.399 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[40346]: purged_snaps scrub ok 2024-09-16T10:31:24.399 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[40346]: osd.3 [v2:172.21.15.164:6826/3793297178,v1:172.21.15.164:6827/3793297178] boot 2024-09-16T10:31:24.399 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[40346]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T10:31:24.399 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:24.399 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[40346]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:24.399 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[25828]: purged_snaps scrub starts 2024-09-16T10:31:24.399 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[25828]: purged_snaps scrub ok 2024-09-16T10:31:24.400 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[25828]: osd.3 [v2:172.21.15.164:6826/3793297178,v1:172.21.15.164:6827/3793297178] boot 2024-09-16T10:31:24.400 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[25828]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T10:31:24.400 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:24.400 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:24 smithi164 ceph-mon[25828]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:24.409 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:24 smithi175 ceph-mon[30341]: purged_snaps scrub starts 2024-09-16T10:31:24.409 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:24 smithi175 ceph-mon[30341]: purged_snaps scrub ok 2024-09-16T10:31:24.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:24 smithi175 ceph-mon[30341]: osd.3 [v2:172.21.15.164:6826/3793297178,v1:172.21.15.164:6827/3793297178] boot 2024-09-16T10:31:24.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:24 smithi175 ceph-mon[30341]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T10:31:24.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:24 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:31:24.410 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:24 smithi175 ceph-mon[30341]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:25.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T10:31:25.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: Detected new or changed devices on smithi164 2024-09-16T10:31:25.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: Adjusting osd_memory_target on smithi164 to 4016M 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:25.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:25.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:25 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T10:31:25.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: Detected new or changed devices on smithi164 2024-09-16T10:31:25.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: Adjusting osd_memory_target on smithi164 to 4016M 2024-09-16T10:31:25.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:25.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:25.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:25.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:25.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:25.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T10:31:25.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: Detected new or changed devices on smithi164 2024-09-16T10:31:25.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:25.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: Adjusting osd_memory_target on smithi164 to 4016M 2024-09-16T10:31:25.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:25.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:25.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:25.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:25.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:25.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:25.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:25 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:26.343 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:26 smithi175 ceph-mon[30341]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:31:26.343 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:31:26.343 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:31:26.343 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:26.343 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:26 smithi175 ceph-mon[30341]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:26.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:26 smithi164 ceph-mon[40346]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:31:26.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:26 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:31:26.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:26 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:31:26.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:26 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:26.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:26 smithi164 ceph-mon[40346]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:26.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:26 smithi164 ceph-mon[25828]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:31:26.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:31:26.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:31:26.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:26.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:26 smithi164 ceph-mon[25828]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:27.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:27 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/4031637777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0745f5b-704f-420b-82de-12d555db0e27"}]: dispatch 2024-09-16T10:31:27.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:27 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0745f5b-704f-420b-82de-12d555db0e27"}]: dispatch 2024-09-16T10:31:27.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:27 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0745f5b-704f-420b-82de-12d555db0e27"}]': finished 2024-09-16T10:31:27.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:27 smithi164 ceph-mon[40346]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T10:31:27.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:27 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:27.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:27 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/4031637777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0745f5b-704f-420b-82de-12d555db0e27"}]: dispatch 2024-09-16T10:31:27.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:27 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0745f5b-704f-420b-82de-12d555db0e27"}]: dispatch 2024-09-16T10:31:27.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:27 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0745f5b-704f-420b-82de-12d555db0e27"}]': finished 2024-09-16T10:31:27.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:27 smithi164 ceph-mon[25828]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T10:31:27.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:27 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:27.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:27 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/4031637777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0745f5b-704f-420b-82de-12d555db0e27"}]: dispatch 2024-09-16T10:31:27.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:27 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0745f5b-704f-420b-82de-12d555db0e27"}]: dispatch 2024-09-16T10:31:27.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:27 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0745f5b-704f-420b-82de-12d555db0e27"}]': finished 2024-09-16T10:31:27.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:27 smithi175 ceph-mon[30341]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T10:31:27.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:27 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:28.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:28 smithi164 ceph-mon[25828]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:28.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:28 smithi164 ceph-mon[40346]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:28.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:28 smithi175 ceph-mon[30341]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:29.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:29 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/390340254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:31:29.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:29 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/390340254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:31:29.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:29 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/390340254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:31:30.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:30 smithi164 ceph-mon[25828]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:30.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:30 smithi164 ceph-mon[40346]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:30.951 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:30 smithi175 ceph-mon[30341]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:32.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:32 smithi175 ceph-mon[30341]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:33.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:32 smithi164 ceph-mon[25828]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:33.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:32 smithi164 ceph-mon[40346]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:34.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:34 smithi175 ceph-mon[30341]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:34.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:34 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T10:31:34.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:34 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:34.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:34 smithi175 ceph-mon[30341]: Deploying daemon osd.4 on smithi175 2024-09-16T10:31:35.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:34 smithi164 ceph-mon[25828]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:35.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:34 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T10:31:35.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:34 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:35.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:34 smithi164 ceph-mon[25828]: Deploying daemon osd.4 on smithi175 2024-09-16T10:31:35.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:34 smithi164 ceph-mon[40346]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:35.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:34 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T10:31:35.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:34 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:35.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:34 smithi164 ceph-mon[40346]: Deploying daemon osd.4 on smithi175 2024-09-16T10:31:36.950 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:36 smithi175 ceph-mon[30341]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:37.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:36 smithi164 ceph-mon[40346]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:37.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:36 smithi164 ceph-mon[25828]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:39.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:38 smithi164 ceph-mon[40346]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:39.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:38 smithi164 ceph-mon[25828]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:39.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:38 smithi175 ceph-mon[30341]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:39.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:39 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:39.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:39 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:39.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:39 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:40.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:39 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:40.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:39 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:40.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:39 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:40.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:39 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:40.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:39 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:40.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:39 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:41.008 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:40 smithi175 ceph-mon[30341]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:41.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:40 smithi164 ceph-mon[40346]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:41.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:40 smithi164 ceph-mon[25828]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:42.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:42 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:42.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:42 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:42.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:42 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:42.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:42 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:42.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:42 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:42.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:42 smithi175 ceph-mon[30341]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:42.345 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:42.345 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:42.345 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:42.345 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:42.345 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:42.345 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[40346]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:42.345 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:42.345 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:42.346 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:42.346 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:42.346 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:42.346 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:42 smithi164 ceph-mon[25828]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:42.430 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 4 on host 'smithi175' 2024-09-16T10:31:43.028 DEBUG:teuthology.orchestra.run.smithi175:osd.4> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T10:31:43.031 INFO:tasks.cephadm:Deploying osd.5 on smithi175 with /dev/nvme2n1... 2024-09-16T10:31:43.031 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-16T10:31:43.293 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:43 smithi175 ceph-mon[30341]: from='osd.4 [v2:172.21.15.175:6800/2959667189,v1:172.21.15.175:6801/2959667189]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T10:31:43.293 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:43 smithi175 ceph-mon[30341]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T10:31:43.293 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:43 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:43.293 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:43 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:43.293 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:43 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:43.373 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:43 smithi164 ceph-mon[40346]: from='osd.4 [v2:172.21.15.175:6800/2959667189,v1:172.21.15.175:6801/2959667189]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T10:31:43.373 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:43 smithi164 ceph-mon[40346]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T10:31:43.373 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:43 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:43.373 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:43 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:43.373 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:43 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:43.373 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:43 smithi164 ceph-mon[25828]: from='osd.4 [v2:172.21.15.175:6800/2959667189,v1:172.21.15.175:6801/2959667189]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T10:31:43.373 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:43 smithi164 ceph-mon[25828]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T10:31:43.373 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:43 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:31:43.374 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:43 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:43.374 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:43 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:43.731 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:31:44.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:44 smithi175 ceph-mon[30341]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T10:31:44.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:44 smithi175 ceph-mon[30341]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T10:31:44.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:44.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:44 smithi175 ceph-mon[30341]: from='osd.4 [v2:172.21.15.175:6800/2959667189,v1:172.21.15.175:6801/2959667189]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:31:44.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:44 smithi175 ceph-mon[30341]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:31:44.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:44 smithi175 ceph-mon[30341]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:44.181 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:31:44 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:31:44.055+0000 7f19db498640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T10:31:44.181 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:31:44 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:31:44.061+0000 7f19d72c1640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T10:31:44.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[25828]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T10:31:44.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[25828]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T10:31:44.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:44.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[25828]: from='osd.4 [v2:172.21.15.175:6800/2959667189,v1:172.21.15.175:6801/2959667189]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:31:44.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[25828]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:31:44.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[25828]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:44.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[40346]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T10:31:44.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[40346]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T10:31:44.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:44.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[40346]: from='osd.4 [v2:172.21.15.175:6800/2959667189,v1:172.21.15.175:6801/2959667189]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:31:44.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[40346]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:31:44.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:44 smithi164 ceph-mon[40346]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-16T10:31:45.219 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:45 smithi175 ceph-mon[30341]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:31:45.219 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:45 smithi175 ceph-mon[30341]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T10:31:45.219 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:45 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:45.219 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:45 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:45.219 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:45 smithi175 ceph-mon[30341]: osd.4 [v2:172.21.15.175:6800/2959667189,v1:172.21.15.175:6801/2959667189] boot 2024-09-16T10:31:45.219 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:45 smithi175 ceph-mon[30341]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T10:31:45.219 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:45 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:45.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[40346]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:31:45.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[40346]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T10:31:45.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:45.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:45.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[40346]: osd.4 [v2:172.21.15.175:6800/2959667189,v1:172.21.15.175:6801/2959667189] boot 2024-09-16T10:31:45.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[40346]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T10:31:45.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:45.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[25828]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:31:45.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[25828]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T10:31:45.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:45.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:45.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[25828]: osd.4 [v2:172.21.15.175:6800/2959667189,v1:172.21.15.175:6801/2959667189] boot 2024-09-16T10:31:45.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[25828]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T10:31:45.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:45 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:31:45.886 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/nvme2n1 2024-09-16T10:31:45.886 INFO:teuthology.orchestra.run.smithi175.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T10:31:45.890 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-16T10:31:45.905 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-09-16T10:31:45.905 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-09-16T10:31:45.905 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131018 s, 800 MB/s 2024-09-16T10:31:45.905 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-09-16T10:31:46.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:46 smithi164 ceph-mon[25828]: purged_snaps scrub starts 2024-09-16T10:31:46.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:46 smithi164 ceph-mon[25828]: purged_snaps scrub ok 2024-09-16T10:31:46.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:46 smithi164 ceph-mon[25828]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T10:31:46.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:46 smithi164 ceph-mon[25828]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:31:46.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:46 smithi164 ceph-mon[40346]: purged_snaps scrub starts 2024-09-16T10:31:46.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:46 smithi164 ceph-mon[40346]: purged_snaps scrub ok 2024-09-16T10:31:46.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:46 smithi164 ceph-mon[40346]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T10:31:46.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:46 smithi164 ceph-mon[40346]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:31:46.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:46 smithi175 ceph-mon[30341]: purged_snaps scrub starts 2024-09-16T10:31:46.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:46 smithi175 ceph-mon[30341]: purged_snaps scrub ok 2024-09-16T10:31:46.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:46 smithi175 ceph-mon[30341]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T10:31:46.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:46 smithi175 ceph-mon[30341]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:31:46.800 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi175:/dev/nvme2n1 2024-09-16T10:31:47.002 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:31:47.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:47 smithi164 ceph-mon[25828]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T10:31:47.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:47 smithi164 ceph-mon[40346]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T10:31:47.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:47 smithi175 ceph-mon[30341]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T10:31:48.188 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:48 smithi175 ceph-mon[30341]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T10:31:48.188 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:48 smithi175 ceph-mon[30341]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:31:48.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:48 smithi164 ceph-mon[25828]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T10:31:48.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:48 smithi164 ceph-mon[25828]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:31:48.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:48 smithi164 ceph-mon[40346]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T10:31:48.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:48 smithi164 ceph-mon[40346]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:31:49.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[25828]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T10:31:49.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[25828]: Detected new or changed devices on smithi175 2024-09-16T10:31:49.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:49.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:49.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:49.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[25828]: Adjusting osd_memory_target on smithi175 to 17088M 2024-09-16T10:31:49.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:49.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:49.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:49.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:49.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[40346]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T10:31:49.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[40346]: Detected new or changed devices on smithi175 2024-09-16T10:31:49.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:49.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:49.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:49.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[40346]: Adjusting osd_memory_target on smithi175 to 17088M 2024-09-16T10:31:49.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:49.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:49.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:49.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:49 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:49.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:49 smithi175 ceph-mon[30341]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T10:31:49.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:49 smithi175 ceph-mon[30341]: Detected new or changed devices on smithi175 2024-09-16T10:31:49.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:49 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:49.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:49 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:49.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:49 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:31:49.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:49 smithi175 ceph-mon[30341]: Adjusting osd_memory_target on smithi175 to 17088M 2024-09-16T10:31:49.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:49 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:49.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:49 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:49.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:49 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:31:49.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:49 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:31:50.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:50 smithi175 ceph-mon[30341]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:31:50.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:50 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:31:50.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:50 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:31:50.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:50 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:50.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:50 smithi175 ceph-mon[30341]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:31:50.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:50 smithi164 ceph-mon[25828]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:31:50.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:50 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:31:50.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:50 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:31:50.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:50 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:50.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:50 smithi164 ceph-mon[25828]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:31:50.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:50 smithi164 ceph-mon[40346]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:31:50.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:50 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:31:50.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:50 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:31:50.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:50 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:50.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:50 smithi164 ceph-mon[40346]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:31:51.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:51 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/57541768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "136e5111-e72a-4acf-aa93-63fe31c75284"}]: dispatch 2024-09-16T10:31:51.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:51 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "136e5111-e72a-4acf-aa93-63fe31c75284"}]: dispatch 2024-09-16T10:31:51.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:51 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "136e5111-e72a-4acf-aa93-63fe31c75284"}]': finished 2024-09-16T10:31:51.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:51 smithi164 ceph-mon[40346]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T10:31:51.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:51 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:31:51.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:51 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/57541768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "136e5111-e72a-4acf-aa93-63fe31c75284"}]: dispatch 2024-09-16T10:31:51.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:51 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "136e5111-e72a-4acf-aa93-63fe31c75284"}]: dispatch 2024-09-16T10:31:51.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:51 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "136e5111-e72a-4acf-aa93-63fe31c75284"}]': finished 2024-09-16T10:31:51.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:51 smithi164 ceph-mon[25828]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T10:31:51.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:51 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:31:51.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:51 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/57541768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "136e5111-e72a-4acf-aa93-63fe31c75284"}]: dispatch 2024-09-16T10:31:51.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:51 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "136e5111-e72a-4acf-aa93-63fe31c75284"}]: dispatch 2024-09-16T10:31:51.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:51 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "136e5111-e72a-4acf-aa93-63fe31c75284"}]': finished 2024-09-16T10:31:51.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:51 smithi175 ceph-mon[30341]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T10:31:51.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:51 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:31:52.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:52 smithi175 ceph-mon[30341]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T10:31:52.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:52 smithi164 ceph-mon[25828]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T10:31:52.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:52 smithi164 ceph-mon[40346]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T10:31:53.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:53 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/2998247173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:31:53.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:53 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/2998247173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:31:53.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:53 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/2998247173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:31:54.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:54 smithi164 ceph-mon[40346]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-16T10:31:54.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:54 smithi164 ceph-mon[25828]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-16T10:31:54.972 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:54 smithi175 ceph-mon[30341]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-16T10:31:56.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:56 smithi175 ceph-mon[30341]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T10:31:57.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:56 smithi164 ceph-mon[25828]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T10:31:57.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:56 smithi164 ceph-mon[40346]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T10:31:59.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:58 smithi175 ceph-mon[30341]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-16T10:31:59.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:58 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T10:31:59.173 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:58 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:59.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:58 smithi164 ceph-mon[25828]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-16T10:31:59.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:58 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T10:31:59.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:58 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:59.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:58 smithi164 ceph-mon[40346]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-16T10:31:59.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:58 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T10:31:59.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:58 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:31:59.963 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:31:59 smithi175 ceph-mon[30341]: Deploying daemon osd.5 on smithi175 2024-09-16T10:32:00.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:31:59 smithi164 ceph-mon[25828]: Deploying daemon osd.5 on smithi175 2024-09-16T10:32:00.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:31:59 smithi164 ceph-mon[40346]: Deploying daemon osd.5 on smithi175 2024-09-16T10:32:00.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:00 smithi175 ceph-mon[30341]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:32:01.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:00 smithi164 ceph-mon[25828]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:32:01.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:00 smithi164 ceph-mon[40346]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:32:03.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:02 smithi164 ceph-mon[25828]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:03.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:02 smithi164 ceph-mon[40346]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:03.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:02 smithi175 ceph-mon[30341]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:03.956 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:03.957 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:03.957 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:03 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:04.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:04.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:04.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:03 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:04.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:04.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:04.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:03 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:05.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:04 smithi175 ceph-mon[30341]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:05.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:04 smithi164 ceph-mon[25828]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:05.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:04 smithi164 ceph-mon[40346]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:05.721 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 5 on host 'smithi175' 2024-09-16T10:32:06.698 DEBUG:teuthology.orchestra.run.smithi175:osd.5> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T10:32:06.700 INFO:tasks.cephadm:Deploying osd.6 on smithi175 with /dev/nvme3n1... 2024-09-16T10:32:06.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-16T10:32:06.936 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:32:06.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:06 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:06.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:06 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:06.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:06 smithi175 ceph-mon[30341]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:06.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:06 smithi175 ceph-mon[30341]: from='osd.5 [v2:172.21.15.175:6808/1222908899,v1:172.21.15.175:6809/1222908899]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T10:32:06.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:06 smithi175 ceph-mon[30341]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T10:32:07.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:06 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:07.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:06 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:07.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:06 smithi164 ceph-mon[40346]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:07.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:06 smithi164 ceph-mon[40346]: from='osd.5 [v2:172.21.15.175:6808/1222908899,v1:172.21.15.175:6809/1222908899]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T10:32:07.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:06 smithi164 ceph-mon[40346]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T10:32:07.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:06 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:07.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:06 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:07.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:06 smithi164 ceph-mon[25828]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:07.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:06 smithi164 ceph-mon[25828]: from='osd.5 [v2:172.21.15.175:6808/1222908899,v1:172.21.15.175:6809/1222908899]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T10:32:07.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:06 smithi164 ceph-mon[25828]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T10:32:07.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:07 smithi175 ceph-mon[30341]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T10:32:07.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:07 smithi175 ceph-mon[30341]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T10:32:07.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:07 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:07.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:07 smithi175 ceph-mon[30341]: from='osd.5 [v2:172.21.15.175:6808/1222908899,v1:172.21.15.175:6809/1222908899]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:07.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:07 smithi175 ceph-mon[30341]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:07.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:07 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:07.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:07 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:07.986 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:32:07 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:32:07.734+0000 7f8a0973b640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T10:32:07.986 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:32:07 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:32:07.739+0000 7f8a05564640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T10:32:08.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[40346]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T10:32:08.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[40346]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T10:32:08.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:08.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[40346]: from='osd.5 [v2:172.21.15.175:6808/1222908899,v1:172.21.15.175:6809/1222908899]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:08.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[40346]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:08.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:08.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:08.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[25828]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T10:32:08.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[25828]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T10:32:08.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:08.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[25828]: from='osd.5 [v2:172.21.15.175:6808/1222908899,v1:172.21.15.175:6809/1222908899]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:08.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[25828]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:08.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:08.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:07 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:08.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:08 smithi175 ceph-mon[30341]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:32:08.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:08 smithi175 ceph-mon[30341]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T10:32:08.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:08 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:08.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:08 smithi175 ceph-mon[30341]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:08.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:08 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:09.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:08 smithi164 ceph-mon[25828]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:32:09.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:08 smithi164 ceph-mon[25828]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T10:32:09.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:09.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:08 smithi164 ceph-mon[25828]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:09.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:08 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:09.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:08 smithi164 ceph-mon[40346]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:32:09.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:08 smithi164 ceph-mon[40346]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T10:32:09.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:08 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:09.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:08 smithi164 ceph-mon[40346]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-16T10:32:09.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:08 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:09.966 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:09 smithi175 ceph-mon[30341]: purged_snaps scrub starts 2024-09-16T10:32:09.966 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:09 smithi175 ceph-mon[30341]: purged_snaps scrub ok 2024-09-16T10:32:09.966 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:09 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:09.966 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:09 smithi175 ceph-mon[30341]: osd.5 [v2:172.21.15.175:6808/1222908899,v1:172.21.15.175:6809/1222908899] boot 2024-09-16T10:32:09.966 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:09 smithi175 ceph-mon[30341]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T10:32:09.966 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:09 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:09.966 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:09 smithi175 ceph-mon[30341]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T10:32:10.065 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/nvme3n1 2024-09-16T10:32:10.065 INFO:teuthology.orchestra.run.smithi175.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T10:32:10.070 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-16T10:32:10.086 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-09-16T10:32:10.086 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-09-16T10:32:10.086 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137501 s, 763 MB/s 2024-09-16T10:32:10.086 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-09-16T10:32:10.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[25828]: purged_snaps scrub starts 2024-09-16T10:32:10.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[25828]: purged_snaps scrub ok 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[25828]: osd.5 [v2:172.21.15.175:6808/1222908899,v1:172.21.15.175:6809/1222908899] boot 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[25828]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[25828]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[40346]: purged_snaps scrub starts 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[40346]: purged_snaps scrub ok 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[40346]: osd.5 [v2:172.21.15.175:6808/1222908899,v1:172.21.15.175:6809/1222908899] boot 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[40346]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:32:10.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:09 smithi164 ceph-mon[40346]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T10:32:10.800 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi175:/dev/nvme3n1 2024-09-16T10:32:11.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:10 smithi164 ceph-mon[25828]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:11.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:10 smithi164 ceph-mon[25828]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T10:32:11.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:10 smithi164 ceph-mon[40346]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:11.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:10 smithi164 ceph-mon[40346]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T10:32:11.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:10 smithi175 ceph-mon[30341]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:11.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:10 smithi175 ceph-mon[30341]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T10:32:11.634 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:32:12.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T10:32:12.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: Detected new or changed devices on smithi175 2024-09-16T10:32:12.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:12.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:12.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: Adjusting osd_memory_target on smithi175 to 8544M 2024-09-16T10:32:12.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:12.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:12.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:12.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:12.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:12.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:12 smithi175 ceph-mon[30341]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:12.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T10:32:12.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: Detected new or changed devices on smithi175 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: Adjusting osd_memory_target on smithi175 to 8544M 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:12.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:12.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[40346]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:12.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T10:32:12.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: Detected new or changed devices on smithi175 2024-09-16T10:32:12.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:12.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:12.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: Adjusting osd_memory_target on smithi175 to 8544M 2024-09-16T10:32:12.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.446 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:12.446 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:12.446 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.446 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:12.446 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:12.446 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:12.446 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:12.446 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:12 smithi164 ceph-mon[25828]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:13.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:13 smithi164 ceph-mon[25828]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T10:32:13.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:13 smithi164 ceph-mon[40346]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T10:32:13.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:13 smithi175 ceph-mon[30341]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T10:32:14.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:14 smithi175 ceph-mon[30341]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:32:14.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:32:14.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:32:14.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:14.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:14 smithi175 ceph-mon[30341]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:14.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:14 smithi164 ceph-mon[40346]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:32:14.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:32:14.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:32:14.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:14.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:14 smithi164 ceph-mon[40346]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:14.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:14 smithi164 ceph-mon[25828]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:32:14.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:32:14.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:32:14.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:14.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:14 smithi164 ceph-mon[25828]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:15.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:15 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/2820850690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766b9300-83a6-4ab4-a0d2-b7cb9e32ed60"}]: dispatch 2024-09-16T10:32:15.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:15 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766b9300-83a6-4ab4-a0d2-b7cb9e32ed60"}]: dispatch 2024-09-16T10:32:15.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:15 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "766b9300-83a6-4ab4-a0d2-b7cb9e32ed60"}]': finished 2024-09-16T10:32:15.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:15 smithi164 ceph-mon[25828]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T10:32:15.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:15 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:15.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:15 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/2820850690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766b9300-83a6-4ab4-a0d2-b7cb9e32ed60"}]: dispatch 2024-09-16T10:32:15.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:15 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766b9300-83a6-4ab4-a0d2-b7cb9e32ed60"}]: dispatch 2024-09-16T10:32:15.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:15 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "766b9300-83a6-4ab4-a0d2-b7cb9e32ed60"}]': finished 2024-09-16T10:32:15.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:15 smithi164 ceph-mon[40346]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T10:32:15.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:15 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:15.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:15 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/2820850690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766b9300-83a6-4ab4-a0d2-b7cb9e32ed60"}]: dispatch 2024-09-16T10:32:15.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:15 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766b9300-83a6-4ab4-a0d2-b7cb9e32ed60"}]: dispatch 2024-09-16T10:32:15.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:15 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "766b9300-83a6-4ab4-a0d2-b7cb9e32ed60"}]': finished 2024-09-16T10:32:15.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:15 smithi175 ceph-mon[30341]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T10:32:15.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:15 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:16.840 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:16 smithi175 ceph-mon[30341]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:16.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:16 smithi164 ceph-mon[25828]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:16.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:16 smithi164 ceph-mon[40346]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:17.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:17 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/4202224842' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:32:17.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:17 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/4202224842' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:32:17.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:17 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/4202224842' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:32:18.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:18 smithi164 ceph-mon[25828]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:18.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:18 smithi164 ceph-mon[40346]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:18.984 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:18 smithi175 ceph-mon[30341]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:21.159 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:20 smithi175 ceph-mon[30341]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:21.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:20 smithi164 ceph-mon[25828]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:21.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:20 smithi164 ceph-mon[40346]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:22.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:22 smithi175 ceph-mon[30341]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:22.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:22 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T10:32:22.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:22 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:23.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:22 smithi164 ceph-mon[25828]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:23.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:22 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T10:32:23.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:22 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:23.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:22 smithi164 ceph-mon[40346]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:23.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:22 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T10:32:23.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:22 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:24.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:23 smithi164 ceph-mon[25828]: Deploying daemon osd.6 on smithi175 2024-09-16T10:32:24.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:23 smithi164 ceph-mon[40346]: Deploying daemon osd.6 on smithi175 2024-09-16T10:32:24.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:23 smithi175 ceph-mon[30341]: Deploying daemon osd.6 on smithi175 2024-09-16T10:32:24.953 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:24 smithi175 ceph-mon[30341]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:25.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:24 smithi164 ceph-mon[40346]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:25.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:24 smithi164 ceph-mon[25828]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:27.066 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:26 smithi175 ceph-mon[30341]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:27.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:26 smithi164 ceph-mon[40346]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:27.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:26 smithi164 ceph-mon[25828]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:27.835 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:27 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:27.836 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:27 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:27.836 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:27 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:28.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:27 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:28.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:27 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:28.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:27 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:28.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:27 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:28.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:27 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:28.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:27 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:28.865 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:28 smithi175 ceph-mon[30341]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:29.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:28 smithi164 ceph-mon[25828]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:29.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:28 smithi164 ceph-mon[40346]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:30.550 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 6 on host 'smithi175' 2024-09-16T10:32:31.080 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:30 smithi175 ceph-mon[30341]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:31.080 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:30 smithi175 ceph-mon[30341]: from='osd.6 [v2:172.21.15.175:6816/2771950113,v1:172.21.15.175:6817/2771950113]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T10:32:31.080 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:30 smithi175 ceph-mon[30341]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T10:32:31.080 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:30 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:31.080 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:30 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:31.099 DEBUG:teuthology.orchestra.run.smithi175:osd.6> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T10:32:31.102 INFO:tasks.cephadm:Deploying osd.7 on smithi175 with /dev/nvme4n1... 2024-09-16T10:32:31.102 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-16T10:32:31.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:30 smithi164 ceph-mon[25828]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:31.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:30 smithi164 ceph-mon[25828]: from='osd.6 [v2:172.21.15.175:6816/2771950113,v1:172.21.15.175:6817/2771950113]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T10:32:31.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:30 smithi164 ceph-mon[25828]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T10:32:31.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:30 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:31.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:30 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:30 smithi164 ceph-mon[40346]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:30 smithi164 ceph-mon[40346]: from='osd.6 [v2:172.21.15.175:6816/2771950113,v1:172.21.15.175:6817/2771950113]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T10:32:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:30 smithi164 ceph-mon[40346]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T10:32:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:31.342 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:32:31.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:31 smithi175 ceph-mon[30341]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T10:32:31.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:31 smithi175 ceph-mon[30341]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T10:32:31.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:31 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:31.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:31 smithi175 ceph-mon[30341]: from='osd.6 [v2:172.21.15.175:6816/2771950113,v1:172.21.15.175:6817/2771950113]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:31.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:31 smithi175 ceph-mon[30341]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:31.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:31 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:31.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:31 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:32.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[25828]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T10:32:32.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[25828]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T10:32:32.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:32.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[25828]: from='osd.6 [v2:172.21.15.175:6816/2771950113,v1:172.21.15.175:6817/2771950113]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:32.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[25828]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:32.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:32.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[40346]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T10:32:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[40346]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T10:32:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[40346]: from='osd.6 [v2:172.21.15.175:6816/2771950113,v1:172.21.15.175:6817/2771950113]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[40346]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:32.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:31 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:32.379 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:32:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:32:32.288+0000 7f7cbb344640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T10:32:32.379 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:32:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:32:32.294+0000 7f7cb3165640 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T10:32:32.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:32 smithi175 ceph-mon[30341]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:32.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:32 smithi175 ceph-mon[30341]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:32:32.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:32 smithi175 ceph-mon[30341]: osdmap e46: 7 total, 6 up, 7 in 2024-09-16T10:32:32.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:32.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:32.906 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:32 smithi175 ceph-mon[30341]: from='osd.6 ' entity='osd.6' 2024-09-16T10:32:33.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[25828]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:33.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[25828]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:32:33.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[25828]: osdmap e46: 7 total, 6 up, 7 in 2024-09-16T10:32:33.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:33.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:33.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[25828]: from='osd.6 ' entity='osd.6' 2024-09-16T10:32:33.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[40346]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:33.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[40346]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:32:33.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[40346]: osdmap e46: 7 total, 6 up, 7 in 2024-09-16T10:32:33.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:33.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:33.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:32 smithi164 ceph-mon[40346]: from='osd.6 ' entity='osd.6' 2024-09-16T10:32:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/nvme4n1 2024-09-16T10:32:33.907 INFO:teuthology.orchestra.run.smithi175.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-16T10:32:33.912 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-16T10:32:33.929 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-09-16T10:32:33.930 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-09-16T10:32:33.930 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0153315 s, 684 MB/s 2024-09-16T10:32:33.930 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-09-16T10:32:34.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[40346]: purged_snaps scrub starts 2024-09-16T10:32:34.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[40346]: purged_snaps scrub ok 2024-09-16T10:32:34.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:34.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[40346]: osd.6 [v2:172.21.15.175:6816/2771950113,v1:172.21.15.175:6817/2771950113] boot 2024-09-16T10:32:34.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[40346]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T10:32:34.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:34.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[25828]: purged_snaps scrub starts 2024-09-16T10:32:34.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[25828]: purged_snaps scrub ok 2024-09-16T10:32:34.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:34.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[25828]: osd.6 [v2:172.21.15.175:6816/2771950113,v1:172.21.15.175:6817/2771950113] boot 2024-09-16T10:32:34.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[25828]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T10:32:34.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:33 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:34.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:33 smithi175 ceph-mon[30341]: purged_snaps scrub starts 2024-09-16T10:32:34.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:33 smithi175 ceph-mon[30341]: purged_snaps scrub ok 2024-09-16T10:32:34.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:33 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:34.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:33 smithi175 ceph-mon[30341]: osd.6 [v2:172.21.15.175:6816/2771950113,v1:172.21.15.175:6817/2771950113] boot 2024-09-16T10:32:34.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:33 smithi175 ceph-mon[30341]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T10:32:34.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:33 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:32:34.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:34 smithi175 ceph-mon[30341]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:34.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:34 smithi175 ceph-mon[30341]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T10:32:35.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:34 smithi164 ceph-mon[40346]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:35.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:34 smithi164 ceph-mon[40346]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T10:32:35.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:34 smithi164 ceph-mon[25828]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-16T10:32:35.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:34 smithi164 ceph-mon[25828]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T10:32:35.217 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi175:/dev/nvme4n1 2024-09-16T10:32:35.429 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:32:36.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:35 smithi175 ceph-mon[30341]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T10:32:36.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:35 smithi164 ceph-mon[25828]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T10:32:36.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:35 smithi164 ceph-mon[40346]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T10:32:36.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:36.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T10:32:36.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:36.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:36.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:36.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:36.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:36.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:36.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:36.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:36.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:36.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:36.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:36.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:37.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:37.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T10:32:37.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:37.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:37.196 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:37.196 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:32:38.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:37 smithi164 ceph-mon[40346]: Detected new or changed devices on smithi175 2024-09-16T10:32:38.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:37 smithi164 ceph-mon[40346]: Adjusting osd_memory_target on smithi175 to 5696M 2024-09-16T10:32:38.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:37 smithi164 ceph-mon[40346]: osdmap e51: 7 total, 7 up, 7 in 2024-09-16T10:32:38.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:37 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:38.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:37 smithi164 ceph-mon[25828]: Detected new or changed devices on smithi175 2024-09-16T10:32:38.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:37 smithi164 ceph-mon[25828]: Adjusting osd_memory_target on smithi175 to 5696M 2024-09-16T10:32:38.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:37 smithi164 ceph-mon[25828]: osdmap e51: 7 total, 7 up, 7 in 2024-09-16T10:32:38.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:37 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:38.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:37 smithi175 ceph-mon[30341]: Detected new or changed devices on smithi175 2024-09-16T10:32:38.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:37 smithi175 ceph-mon[30341]: Adjusting osd_memory_target on smithi175 to 5696M 2024-09-16T10:32:38.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:37 smithi175 ceph-mon[30341]: osdmap e51: 7 total, 7 up, 7 in 2024-09-16T10:32:38.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:37 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:39.159 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:38 smithi175 ceph-mon[30341]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:39.159 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:38 smithi175 ceph-mon[30341]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:32:39.160 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:38 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:32:39.160 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:38 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:32:39.160 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:38 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:39.196 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:38 smithi164 ceph-mon[25828]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:39.196 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:38 smithi164 ceph-mon[25828]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:32:39.196 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:38 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:32:39.197 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:38 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:32:39.197 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:38 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:39.197 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:38 smithi164 ceph-mon[40346]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:39.197 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:38 smithi164 ceph-mon[40346]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:32:39.197 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:38 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T10:32:39.197 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:38 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T10:32:39.197 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:38 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:41.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[25828]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-16T10:32:41.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/1194113964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9dadc7-445b-4e0c-b8a5-1003634eb9d5"}]: dispatch 2024-09-16T10:32:41.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9dadc7-445b-4e0c-b8a5-1003634eb9d5"}]: dispatch 2024-09-16T10:32:41.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[25828]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a9dadc7-445b-4e0c-b8a5-1003634eb9d5"}]': finished 2024-09-16T10:32:41.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[25828]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T10:32:41.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:41.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[40346]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-16T10:32:41.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/1194113964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9dadc7-445b-4e0c-b8a5-1003634eb9d5"}]: dispatch 2024-09-16T10:32:41.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9dadc7-445b-4e0c-b8a5-1003634eb9d5"}]: dispatch 2024-09-16T10:32:41.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[40346]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a9dadc7-445b-4e0c-b8a5-1003634eb9d5"}]': finished 2024-09-16T10:32:41.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[40346]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T10:32:41.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:40 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:41.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:40 smithi175 ceph-mon[30341]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-16T10:32:41.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:40 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/1194113964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9dadc7-445b-4e0c-b8a5-1003634eb9d5"}]: dispatch 2024-09-16T10:32:41.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:40 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a9dadc7-445b-4e0c-b8a5-1003634eb9d5"}]: dispatch 2024-09-16T10:32:41.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:40 smithi175 ceph-mon[30341]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a9dadc7-445b-4e0c-b8a5-1003634eb9d5"}]': finished 2024-09-16T10:32:41.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:40 smithi175 ceph-mon[30341]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T10:32:41.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:40 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:42.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:41 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/3221250636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:32:42.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:41 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/3221250636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:32:42.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:41 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/3221250636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T10:32:43.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:42 smithi164 ceph-mon[40346]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T10:32:43.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:42 smithi164 ceph-mon[25828]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T10:32:43.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:42 smithi175 ceph-mon[30341]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T10:32:44.984 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:44 smithi175 ceph-mon[30341]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-16T10:32:45.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:44 smithi164 ceph-mon[40346]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-16T10:32:45.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:44 smithi164 ceph-mon[25828]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-16T10:32:46.955 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:46 smithi175 ceph-mon[30341]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-16T10:32:47.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:46 smithi164 ceph-mon[25828]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-16T10:32:47.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:46 smithi164 ceph-mon[40346]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-16T10:32:48.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:47 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T10:32:48.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:47 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:48.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:47 smithi164 ceph-mon[40346]: Deploying daemon osd.7 on smithi175 2024-09-16T10:32:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T10:32:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:47 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:48.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:47 smithi164 ceph-mon[25828]: Deploying daemon osd.7 on smithi175 2024-09-16T10:32:48.202 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:47 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T10:32:48.202 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:47 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:32:48.202 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:47 smithi175 ceph-mon[30341]: Deploying daemon osd.7 on smithi175 2024-09-16T10:32:49.075 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:48 smithi175 ceph-mon[30341]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:32:49.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:48 smithi164 ceph-mon[25828]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:32:49.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:48 smithi164 ceph-mon[40346]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:32:51.091 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:50 smithi175 ceph-mon[30341]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:51.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:50 smithi164 ceph-mon[25828]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:51.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:50 smithi164 ceph-mon[40346]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:51 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:51 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:51 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:52.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:51 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:52.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:51 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:52.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:51 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:52.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:51 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:32:52.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:51 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:52.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:51 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:53.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:52 smithi164 ceph-mon[25828]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:53.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:52 smithi164 ceph-mon[40346]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:53.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:52 smithi175 ceph-mon[30341]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:55.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:54 smithi175 ceph-mon[30341]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:55.147 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 7 on host 'smithi175' 2024-09-16T10:32:55.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:54 smithi164 ceph-mon[25828]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:55.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:54 smithi164 ceph-mon[40346]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:55.846 DEBUG:teuthology.orchestra.run.smithi175:osd.7> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T10:32:55.849 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-16T10:32:55.849 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T10:32:56.061 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:32:56.166 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[40346]: from='osd.7 [v2:172.21.15.175:6824/1006272747,v1:172.21.15.175:6825/1006272747]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T10:32:56.166 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[40346]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T10:32:56.166 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:56.166 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:56.166 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:56.166 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:56.166 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[25828]: from='osd.7 [v2:172.21.15.175:6824/1006272747,v1:172.21.15.175:6825/1006272747]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T10:32:56.167 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[25828]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T10:32:56.167 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:56.167 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:56.167 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:56.167 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:55 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:55 smithi175 ceph-mon[30341]: from='osd.7 [v2:172.21.15.175:6824/1006272747,v1:172.21.15.175:6825/1006272747]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T10:32:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:55 smithi175 ceph-mon[30341]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T10:32:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:55 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:55 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:56.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:55 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:56.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:55 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:32:57.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[25828]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:57.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[25828]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T10:32:57.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[25828]: osdmap e53: 8 total, 7 up, 8 in 2024-09-16T10:32:57.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:57.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[25828]: from='osd.7 [v2:172.21.15.175:6824/1006272747,v1:172.21.15.175:6825/1006272747]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:57.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[25828]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:57.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[40346]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:57.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[40346]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T10:32:57.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[40346]: osdmap e53: 8 total, 7 up, 8 in 2024-09-16T10:32:57.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:57.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[40346]: from='osd.7 [v2:172.21.15.175:6824/1006272747,v1:172.21.15.175:6825/1006272747]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:57.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:56 smithi164 ceph-mon[40346]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:57.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:57.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T10:32:57.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: osdmap e53: 8 total, 7 up, 8 in 2024-09-16T10:32:57.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:57.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: from='osd.7 [v2:172.21.15.175:6824/1006272747,v1:172.21.15.175:6825/1006272747]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:57.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-16T10:32:57.538 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:32:57.417+0000 7fbac0076640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T10:32:57.538 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:32:57.426+0000 7fbab7e97640 -1 osd.7 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T10:32:57.884 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:32:58.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[25828]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:32:58.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[25828]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T10:32:58.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:58.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:58.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[25828]: from='osd.7 ' entity='osd.7' 2024-09-16T10:32:58.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1806852432' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T10:32:58.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:58.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[40346]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:32:58.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[40346]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T10:32:58.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:58.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:58.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[40346]: from='osd.7 ' entity='osd.7' 2024-09-16T10:32:58.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1806852432' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T10:32:58.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:57 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-16T10:32:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T10:32:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: from='osd.7 ' entity='osd.7' 2024-09-16T10:32:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1806852432' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T10:32:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:57 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:58.676 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":54,"num_osds":8,"num_up_osds":7,"osd_up_since":1726482752,"num_in_osds":8,"osd_in_since":1726482760,"num_remapped_pgs":0} 2024-09-16T10:32:59.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:58 smithi175 ceph-mon[30341]: purged_snaps scrub starts 2024-09-16T10:32:59.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:58 smithi175 ceph-mon[30341]: purged_snaps scrub ok 2024-09-16T10:32:59.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:58 smithi175 ceph-mon[30341]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:59.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:58 smithi175 ceph-mon[30341]: osd.7 [v2:172.21.15.175:6824/1006272747,v1:172.21.15.175:6825/1006272747] boot 2024-09-16T10:32:59.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:58 smithi175 ceph-mon[30341]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T10:32:59.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:58 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:59.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[40346]: purged_snaps scrub starts 2024-09-16T10:32:59.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[40346]: purged_snaps scrub ok 2024-09-16T10:32:59.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[40346]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:59.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[40346]: osd.7 [v2:172.21.15.175:6824/1006272747,v1:172.21.15.175:6825/1006272747] boot 2024-09-16T10:32:59.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[40346]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T10:32:59.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:59.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[25828]: purged_snaps scrub starts 2024-09-16T10:32:59.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[25828]: purged_snaps scrub ok 2024-09-16T10:32:59.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[25828]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-16T10:32:59.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[25828]: osd.7 [v2:172.21.15.175:6824/1006272747,v1:172.21.15.175:6825/1006272747] boot 2024-09-16T10:32:59.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[25828]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T10:32:59.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:58 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:32:59.678 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T10:32:59.902 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:00.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T10:33:00.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:00.247 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.248 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.249 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.249 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:00.249 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:00.249 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.249 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:00.249 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:00.249 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:00.249 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:32:59 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T10:33:00.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:32:59 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:01.348 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:01 smithi164 ceph-mon[40346]: Detected new or changed devices on smithi175 2024-09-16T10:33:01.348 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:01 smithi164 ceph-mon[40346]: Adjusting osd_memory_target on smithi175 to 4272M 2024-09-16T10:33:01.348 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:01 smithi164 ceph-mon[40346]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:01.348 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:01 smithi164 ceph-mon[40346]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T10:33:01.349 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:01 smithi164 ceph-mon[25828]: Detected new or changed devices on smithi175 2024-09-16T10:33:01.349 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:01 smithi164 ceph-mon[25828]: Adjusting osd_memory_target on smithi175 to 4272M 2024-09-16T10:33:01.349 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:01 smithi164 ceph-mon[25828]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:01.349 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:01 smithi164 ceph-mon[25828]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T10:33:01.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:01 smithi175 ceph-mon[30341]: Detected new or changed devices on smithi175 2024-09-16T10:33:01.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:01 smithi175 ceph-mon[30341]: Adjusting osd_memory_target on smithi175 to 4272M 2024-09-16T10:33:01.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:01 smithi175 ceph-mon[30341]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:01.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:01 smithi175 ceph-mon[30341]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T10:33:01.720 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:33:02.349 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:02 smithi164 ceph-mon[40346]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T10:33:02.349 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:02 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1266088347' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T10:33:02.349 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:02 smithi164 ceph-mon[25828]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T10:33:02.349 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:02 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1266088347' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T10:33:02.370 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1726482777,"num_in_osds":8,"osd_in_since":1726482760,"num_remapped_pgs":0} 2024-09-16T10:33:02.370 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T10:33:02.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:02 smithi175 ceph-mon[30341]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T10:33:02.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:02 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1266088347' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T10:33:02.563 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:03.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:03 smithi164 ceph-mon[25828]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:03.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:03 smithi164 ceph-mon[25828]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T10:33:03.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:03 smithi164 ceph-mon[40346]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:03.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:03 smithi164 ceph-mon[40346]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T10:33:03.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:03 smithi175 ceph-mon[30341]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:03.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:03 smithi175 ceph-mon[30341]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T10:33:04.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:04 smithi164 ceph-mon[25828]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:04.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:04 smithi164 ceph-mon[40346]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:33:04.360 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":59,"fsid":"3d5087c4-7416-11ef-bceb-c7b262605968","created":"2024-09-16T10:27:53.771794+0000","modified":"2024-09-16T10:33:02.000307+0000","last_up_change":"2024-09-16T10:32:57.974199+0000","last_in_change":"2024-09-16T10:32:40.215908+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T10:30:59.996499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8ae783-a95a-4a6e-ad9b-fa68ce6256a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6803","nonce":732894384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6805","nonce":732894384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6809","nonce":732894384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6807","nonce":732894384}]},"public_addr":"172.21.15.164:6803/732894384","cluster_addr":"172.21.15.164:6805/732894384","heartbeat_back_addr":"172.21.15.164:6809/732894384","heartbeat_front_addr":"172.21.15.164:6807/732894384","state":["exists","up"]},{"osd":1,"uuid":"8da925d4-94e1-4f33-9c3f-90d6ea8574ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6811","nonce":4121909442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6813","nonce":4121909442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6817","nonce":4121909442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6815","nonce":4121909442}]},"public_addr":"172.21.15.164:6811/4121909442","cluster_addr":"172.21.15.164:6813/4121909442","heartbeat_back_addr":"172.21.15.164:6817/4121909442","heartbeat_front_addr":"172.21.15.164:6815/4121909442","state":["exists","up"]},{"osd":2,"uuid":"c8c3465b-81e1-4ef4-8574-049b2b8e7455","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6819","nonce":254163173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6821","nonce":254163173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6825","nonce":254163173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6823","nonce":254163173}]},"public_addr":"172.21.15.164:6819/254163173","cluster_addr":"172.21.15.164:6821/254163173","heartbeat_back_addr":"172.21.15.164:6825/254163173","heartbeat_front_addr":"172.21.15.164:6823/254163173","state":["exists","up"]},{"osd":3,"uuid":"0662cc54-7dac-48dd-8802-8f82c6424867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6827","nonce":3793297178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6829","nonce":3793297178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6833","nonce":3793297178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6831","nonce":3793297178}]},"public_addr":"172.21.15.164:6827/3793297178","cluster_addr":"172.21.15.164:6829/3793297178","heartbeat_back_addr":"172.21.15.164:6833/3793297178","heartbeat_front_addr":"172.21.15.164:6831/3793297178","state":["exists","up"]},{"osd":4,"uuid":"c0745f5b-704f-420b-82de-12d555db0e27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6801","nonce":2959667189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6803","nonce":2959667189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6807","nonce":2959667189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6805","nonce":2959667189}]},"public_addr":"172.21.15.175:6801/2959667189","cluster_addr":"172.21.15.175:6803/2959667189","heartbeat_back_addr":"172.21.15.175:6807/2959667189","heartbeat_front_addr":"172.21.15.175:6805/2959667189","state":["exists","up"]},{"osd":5,"uuid":"136e5111-e72a-4acf-aa93-63fe31c75284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6809","nonce":1222908899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6811","nonce":1222908899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6815","nonce":1222908899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6813","nonce":1222908899}]},"public_addr":"172.21.15.175:6809/1222908899","cluster_addr":"172.21.15.175:6811/1222908899","heartbeat_back_addr":"172.21.15.175:6815/1222908899","heartbeat_front_addr":"172.21.15.175:6813/1222908899","state":["exists","up"]},{"osd":6,"uuid":"766b9300-83a6-4ab4-a0d2-b7cb9e32ed60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6817","nonce":2771950113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6819","nonce":2771950113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6823","nonce":2771950113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6821","nonce":2771950113}]},"public_addr":"172.21.15.175:6817/2771950113","cluster_addr":"172.21.15.175:6819/2771950113","heartbeat_back_addr":"172.21.15.175:6823/2771950113","heartbeat_front_addr":"172.21.15.175:6821/2771950113","state":["exists","up"]},{"osd":7,"uuid":"0a9dadc7-445b-4e0c-b8a5-1003634eb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6825","nonce":1006272747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6827","nonce":1006272747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6831","nonce":1006272747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6829","nonce":1006272747}]},"public_addr":"172.21.15.175:6825/1006272747","cluster_addr":"172.21.15.175:6827/1006272747","heartbeat_back_addr":"172.21.15.175:6831/1006272747","heartbeat_front_addr":"172.21.15.175:6829/1006272747","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:30:08.753272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:30:32.724150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:30:56.799127+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:31:21.216920+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:31:43.058730+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:32:07.087680+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:32:31.368288+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:32:56.035998+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/699939050":"2024-09-17T10:28:47.696520+0000","172.21.15.164:6801/3165220513":"2024-09-17T10:28:47.696520+0000","172.21.15.164:0/1934115177":"2024-09-17T10:28:47.696520+0000","172.21.15.164:0/430719755":"2024-09-17T10:28:47.696520+0000","172.21.15.164:6801/2694972888":"2024-09-17T10:28:24.597753+0000","172.21.15.164:6800/2694972888":"2024-09-17T10:28:24.597753+0000","172.21.15.164:0/3996721572":"2024-09-17T10:28:24.597753+0000","172.21.15.164:6800/3165220513":"2024-09-17T10:28:47.696520+0000","172.21.15.164:0/3110320769":"2024-09-17T10:28:24.597753+0000","172.21.15.164:0/3711213828":"2024-09-17T10:28:24.597753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T10:33:04.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:04 smithi175 ceph-mon[30341]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:05.159 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T10:30:59.996499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T10:33:05.160 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T10:33:05.355 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:05.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:05 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/2427012041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T10:33:05.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:05 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/2427012041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T10:33:05.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:05 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/2427012041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T10:33:06.142 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:06 smithi164 ceph-mon[40346]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T10:33:06.142 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:06 smithi164 ceph-mon[25828]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T10:33:06.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:06 smithi175 ceph-mon[30341]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T10:33:07.048 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-09-16T10:33:07.381 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:07 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/832612607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T10:33:07.381 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:07 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/832612607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T10:33:07.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:07 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/832612607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T10:33:07.702 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi164 2024-09-16T10:33:07.703 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi164=foo.a' 2024-09-16T10:33:07.911 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:33:08.297 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:08 smithi175 ceph-mon[30341]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T10:33:08.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:08 smithi164 ceph-mon[40346]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T10:33:08.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:08 smithi164 ceph-mon[25828]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T10:33:09.719 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled rgw.foo.a update... 2024-09-16T10:33:10.477 DEBUG:teuthology.orchestra.run.smithi164:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-16T10:33:10.479 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi175 2024-09-16T10:33:10.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-16T10:33:10.670 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:33:10.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi164=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:10.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: Saving service rgw.foo.a spec with placement smithi164=foo.a;count:1 2024-09-16T10:33:10.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:10.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:10.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:10.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:10.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:10.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T10:33:10.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T10:33:10.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:10.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:10.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: Deploying daemon rgw.foo.a on smithi164 2024-09-16T10:33:10.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:10 smithi175 ceph-mon[30341]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-16T10:33:11.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi164=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:11.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: Saving service rgw.foo.a spec with placement smithi164=foo.a;count:1 2024-09-16T10:33:11.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:11.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:11.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:11.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:11.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:11.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T10:33:11.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T10:33:11.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:11.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:11.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: Deploying daemon rgw.foo.a on smithi164 2024-09-16T10:33:11.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[25828]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-16T10:33:11.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi164=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:11.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: Saving service rgw.foo.a spec with placement smithi164=foo.a;count:1 2024-09-16T10:33:11.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:11.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:11.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:11.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:11.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:11.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T10:33:11.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T10:33:11.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:11.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:11.196 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: Deploying daemon rgw.foo.a on smithi164 2024-09-16T10:33:11.196 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:10 smithi164 ceph-mon[40346]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-16T10:33:11.912 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:33:11 smithi164 systemd[1]: Starting Ceph rgw.foo.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:33:12.193 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:33:11 smithi164 podman[76150]: 2024-09-16 10:33:11.81294496 +0000 UTC m=+0.016150277 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T10:33:12.193 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:33:11 smithi164 podman[76150]: 2024-09-16 10:33:11.913987597 +0000 UTC m=+0.117192942 container create 79aded71f0dc1566d24204977fd27c5ced2a1cd1a0a09604ef9950aafe84ad0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-rgw-foo-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T10:33:12.193 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:33:12 smithi164 podman[76150]: 2024-09-16 10:33:12.149811388 +0000 UTC m=+0.353016701 container init 79aded71f0dc1566d24204977fd27c5ced2a1cd1a0a09604ef9950aafe84ad0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-rgw-foo-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T10:33:12.194 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:33:12 smithi164 podman[76150]: 2024-09-16 10:33:12.154909102 +0000 UTC m=+0.358114406 container start 79aded71f0dc1566d24204977fd27c5ced2a1cd1a0a09604ef9950aafe84ad0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-16T10:33:12.693 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:33:12 smithi164 bash[76150]: 79aded71f0dc1566d24204977fd27c5ced2a1cd1a0a09604ef9950aafe84ad0d 2024-09-16T10:33:12.693 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:33:12 smithi164 systemd[1]: Started Ceph rgw.foo.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:33:12.813 INFO:teuthology.orchestra.run.smithi175.stderr:pool 'datapool' created 2024-09-16T10:33:13.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:12 smithi175 ceph-mon[30341]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:33:13.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:12 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/4174042324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T10:33:13.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:12 smithi175 ceph-mon[30341]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T10:33:13.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:12 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:13.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[25828]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:33:13.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/4174042324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T10:33:13.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[25828]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T10:33:13.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:13.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[40346]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:33:13.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/4174042324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T10:33:13.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[40346]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-16T10:33:13.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:13.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:12 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:13.485 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- rbd pool init datapool 2024-09-16T10:33:13.679 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:33:14.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:13 smithi164 ceph-mon[40346]: Saving service rgw.foo.a spec with placement smithi164=foo.a;count:1 2024-09-16T10:33:14.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:13 smithi164 ceph-mon[40346]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T10:33:14.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:13 smithi164 ceph-mon[40346]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T10:33:14.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:13 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/3176396851' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T10:33:14.072 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:13 smithi164 ceph-mon[40346]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T10:33:14.073 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:13 smithi164 ceph-mon[25828]: Saving service rgw.foo.a spec with placement smithi164=foo.a;count:1 2024-09-16T10:33:14.073 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:13 smithi164 ceph-mon[25828]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T10:33:14.073 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:13 smithi164 ceph-mon[25828]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T10:33:14.073 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:13 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3176396851' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T10:33:14.073 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:13 smithi164 ceph-mon[25828]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T10:33:14.074 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:13 smithi175 ceph-mon[30341]: Saving service rgw.foo.a spec with placement smithi164=foo.a;count:1 2024-09-16T10:33:14.074 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:13 smithi175 ceph-mon[30341]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T10:33:14.074 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:13 smithi175 ceph-mon[30341]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T10:33:14.075 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:13 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/3176396851' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T10:33:14.075 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:13 smithi175 ceph-mon[30341]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T10:33:15.070 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:14 smithi175 ceph-mon[30341]: pgmap v183: 36 pgs: 4 creating+peering, 31 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:33:15.070 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:14 smithi175 ceph-mon[30341]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T10:33:15.070 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:14 smithi175 ceph-mon[30341]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T10:33:15.070 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:14 smithi175 ceph-mon[30341]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T10:33:15.071 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:15.071 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:15.071 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:15.071 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:14 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:15.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[40346]: pgmap v183: 36 pgs: 4 creating+peering, 31 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:33:15.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[40346]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T10:33:15.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[40346]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T10:33:15.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[40346]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[25828]: pgmap v183: 36 pgs: 4 creating+peering, 31 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[25828]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[25828]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[25828]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:15.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:14 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:16.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[40346]: Checking dashboard <-> RGW credentials 2024-09-16T10:33:16.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[40346]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T10:33:16.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1584240436' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[40346]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/3176396851' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[40346]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/3038189322' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T10:33:16.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[40346]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T10:33:16.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[25828]: Checking dashboard <-> RGW credentials 2024-09-16T10:33:16.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[25828]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T10:33:16.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1584240436' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[25828]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3176396851' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.196 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[25828]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.196 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/3038189322' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T10:33:16.196 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:15 smithi164 ceph-mon[25828]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T10:33:16.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:15 smithi175 ceph-mon[30341]: Checking dashboard <-> RGW credentials 2024-09-16T10:33:16.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:15 smithi175 ceph-mon[30341]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T10:33:16.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:15 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1584240436' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:15 smithi175 ceph-mon[30341]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:15 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/3176396851' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:15 smithi175 ceph-mon[30341]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-16T10:33:16.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:15 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/3038189322' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T10:33:16.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:15 smithi175 ceph-mon[30341]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-16T10:33:17.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:16 smithi164 ceph-mon[25828]: pgmap v186: 68 pgs: 24 active+clean, 5 creating+activating, 18 creating+peering, 21 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T10:33:17.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:16 smithi164 ceph-mon[25828]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T10:33:17.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:16 smithi164 ceph-mon[25828]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T10:33:17.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:16 smithi164 ceph-mon[25828]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T10:33:17.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:16 smithi164 ceph-mon[25828]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T10:33:17.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:16 smithi164 ceph-mon[40346]: pgmap v186: 68 pgs: 24 active+clean, 5 creating+activating, 18 creating+peering, 21 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T10:33:17.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:16 smithi164 ceph-mon[40346]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T10:33:17.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:16 smithi164 ceph-mon[40346]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T10:33:17.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:16 smithi164 ceph-mon[40346]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T10:33:17.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:16 smithi164 ceph-mon[40346]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T10:33:17.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:16 smithi175 ceph-mon[30341]: pgmap v186: 68 pgs: 24 active+clean, 5 creating+activating, 18 creating+peering, 21 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T10:33:17.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:16 smithi175 ceph-mon[30341]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T10:33:17.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:16 smithi175 ceph-mon[30341]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T10:33:17.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:16 smithi175 ceph-mon[30341]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T10:33:17.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:16 smithi175 ceph-mon[30341]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T10:33:18.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:17 smithi175 ceph-mon[30341]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T10:33:18.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:17 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/3176396851' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:17 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1584240436' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:17 smithi175 ceph-mon[30341]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.167 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:17 smithi175 ceph-mon[30341]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:17 smithi164 ceph-mon[25828]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T10:33:18.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:17 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3176396851' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:17 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1584240436' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:17 smithi164 ceph-mon[25828]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:17 smithi164 ceph-mon[25828]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:17 smithi164 ceph-mon[40346]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T10:33:18.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:17 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/3176396851' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:17 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1584240436' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:17 smithi164 ceph-mon[40346]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:17 smithi164 ceph-mon[40346]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-16T10:33:18.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.175 --placement '1;smithi175=iscsi.a' 2024-09-16T10:33:18.654 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:33:19.077 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:18 smithi175 ceph-mon[30341]: pgmap v189: 100 pgs: 32 active+clean, 5 creating+activating, 18 creating+peering, 45 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-16T10:33:19.077 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:18 smithi175 ceph-mon[30341]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T10:33:19.077 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:18 smithi175 ceph-mon[30341]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T10:33:19.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:18 smithi175 ceph-mon[30341]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T10:33:19.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:18 smithi164 ceph-mon[40346]: pgmap v189: 100 pgs: 32 active+clean, 5 creating+activating, 18 creating+peering, 45 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-16T10:33:19.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:18 smithi164 ceph-mon[40346]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T10:33:19.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:18 smithi164 ceph-mon[40346]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T10:33:19.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:18 smithi164 ceph-mon[40346]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T10:33:19.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:18 smithi164 ceph-mon[25828]: pgmap v189: 100 pgs: 32 active+clean, 5 creating+activating, 18 creating+peering, 45 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-16T10:33:19.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:18 smithi164 ceph-mon[25828]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T10:33:19.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:18 smithi164 ceph-mon[25828]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T10:33:19.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:18 smithi164 ceph-mon[25828]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T10:33:19.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:19 smithi175 ceph-mon[30341]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T10:33:19.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:19 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1990072501' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:19.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:19 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/964498804' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:19.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:19 smithi175 ceph-mon[30341]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:19.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:19 smithi175 ceph-mon[30341]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:19.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:19 smithi175 ceph-mon[30341]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T10:33:20.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[40346]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T10:33:20.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1990072501' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:20.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/964498804' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:20.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[40346]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:20.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[40346]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:20.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[40346]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T10:33:20.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[25828]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T10:33:20.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1990072501' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:20.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/964498804' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:20.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[25828]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:20.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[25828]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T10:33:20.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:19 smithi164 ceph-mon[25828]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T10:33:20.375 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled iscsi.datapool update... 2024-09-16T10:33:20.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:20 smithi175 ceph-mon[30341]: pgmap v192: 132 pgs: 75 active+clean, 24 creating+peering, 33 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-16T10:33:20.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:20 smithi175 ceph-mon[30341]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T10:33:20.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:20 smithi175 ceph-mon[30341]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T10:33:20.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:20 smithi175 ceph-mon[30341]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T10:33:20.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:20 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/964498804' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:20.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:20 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1990072501' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:20.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:20 smithi175 ceph-mon[30341]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:20.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:20 smithi175 ceph-mon[30341]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:20.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:20 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:21.035 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-16T10:33:21.035 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:33:21.035 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-16T10:33:21.073 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:33:21.073 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-16T10:33:21.102 DEBUG:teuthology.orchestra.run.smithi175:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@iscsi.iscsi.a.service 2024-09-16T10:33:21.146 INFO:tasks.cephadm:Adding prometheus.a on smithi175 2024-09-16T10:33:21.146 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi175=a' 2024-09-16T10:33:21.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[40346]: pgmap v192: 132 pgs: 75 active+clean, 24 creating+peering, 33 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-16T10:33:21.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[40346]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T10:33:21.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[40346]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T10:33:21.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[40346]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T10:33:21.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/964498804' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:21.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1990072501' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:21.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[40346]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:21.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[40346]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:21.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:21.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[25828]: pgmap v192: 132 pgs: 75 active+clean, 24 creating+peering, 33 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-16T10:33:21.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[25828]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T10:33:21.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[25828]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T10:33:21.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[25828]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T10:33:21.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/964498804' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:21.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1990072501' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:21.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[25828]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:21.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[25828]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T10:33:21.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:20 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:22.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[40346]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.175", "placement": "1;smithi175=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:22.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[40346]: Saving service iscsi.datapool spec with placement smithi175=iscsi.a;count:1 2024-09-16T10:33:22.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[40346]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T10:33:22.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[40346]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T10:33:22.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[40346]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T10:33:22.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:22.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:22.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:22.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T10:33:22.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T10:33:22.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:22.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[25828]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.175", "placement": "1;smithi175=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:22.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[25828]: Saving service iscsi.datapool spec with placement smithi175=iscsi.a;count:1 2024-09-16T10:33:22.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[25828]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T10:33:22.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[25828]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T10:33:22.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[25828]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T10:33:22.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:22.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:22.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:22.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T10:33:22.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T10:33:22.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:21 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:22.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:21 smithi175 ceph-mon[30341]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.175", "placement": "1;smithi175=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:22.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:21 smithi175 ceph-mon[30341]: Saving service iscsi.datapool spec with placement smithi175=iscsi.a;count:1 2024-09-16T10:33:22.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:21 smithi175 ceph-mon[30341]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T10:33:22.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:21 smithi175 ceph-mon[30341]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T10:33:22.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:21 smithi175 ceph-mon[30341]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T10:33:22.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:22.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:22.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:22.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T10:33:22.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T10:33:22.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:21 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:22.288 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:33:22.971 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:22 smithi175 ceph-mon[30341]: Deploying daemon iscsi.iscsi.a on smithi175 2024-09-16T10:33:22.972 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:22 smithi175 ceph-mon[30341]: pgmap v195: 132 pgs: 113 active+clean, 10 creating+peering, 9 unknown; 579 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 2.5 KiB/s wr, 47 op/s 2024-09-16T10:33:22.972 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:22 smithi175 ceph-mon[30341]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T10:33:22.972 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:22 smithi175 ceph-mon[30341]: Cluster is now healthy 2024-09-16T10:33:23.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:22 smithi164 ceph-mon[40346]: Deploying daemon iscsi.iscsi.a on smithi175 2024-09-16T10:33:23.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:22 smithi164 ceph-mon[40346]: pgmap v195: 132 pgs: 113 active+clean, 10 creating+peering, 9 unknown; 579 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 2.5 KiB/s wr, 47 op/s 2024-09-16T10:33:23.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:22 smithi164 ceph-mon[40346]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T10:33:23.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:22 smithi164 ceph-mon[40346]: Cluster is now healthy 2024-09-16T10:33:23.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:22 smithi164 ceph-mon[25828]: Deploying daemon iscsi.iscsi.a on smithi175 2024-09-16T10:33:23.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:22 smithi164 ceph-mon[25828]: pgmap v195: 132 pgs: 113 active+clean, 10 creating+peering, 9 unknown; 579 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 2.5 KiB/s wr, 47 op/s 2024-09-16T10:33:23.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:22 smithi164 ceph-mon[25828]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T10:33:23.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:22 smithi164 ceph-mon[25828]: Cluster is now healthy 2024-09-16T10:33:24.738 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled prometheus update... 2024-09-16T10:33:24.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:24 smithi175 ceph-mon[30341]: pgmap v196: 132 pgs: 120 active+clean, 10 creating+peering, 2 unknown; 579 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 2.7 KiB/s wr, 68 op/s 2024-09-16T10:33:24.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:24 smithi175 ceph-mon[30341]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T10:33:24.986 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:24 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:25.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:24 smithi164 ceph-mon[40346]: pgmap v196: 132 pgs: 120 active+clean, 10 creating+peering, 2 unknown; 579 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 2.7 KiB/s wr, 68 op/s 2024-09-16T10:33:25.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:24 smithi164 ceph-mon[40346]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T10:33:25.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:24 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:25.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:24 smithi164 ceph-mon[25828]: pgmap v196: 132 pgs: 120 active+clean, 10 creating+peering, 2 unknown; 579 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 2.7 KiB/s wr, 68 op/s 2024-09-16T10:33:25.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:24 smithi164 ceph-mon[25828]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T10:33:25.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:24 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:25.298 DEBUG:teuthology.orchestra.run.smithi175:prometheus.a> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-16T10:33:25.300 INFO:tasks.cephadm:Adding node-exporter.a on smithi164 2024-09-16T10:33:25.300 INFO:tasks.cephadm:Adding node-exporter.b on smithi175 2024-09-16T10:33:25.301 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '2;smithi164=a;smithi175=b' 2024-09-16T10:33:25.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:25 smithi175 podman[67019]: 2024-09-16 10:33:25.746131498 +0000 UTC m=+0.371997113 container create 1e9a78c1fad707560118af6790e7a697b74511fcc50d2f9bd2568ba15cd46e69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T10:33:26.077 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:33:26.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:25 smithi164 ceph-mon[40346]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:26.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:25 smithi164 ceph-mon[40346]: Saving service prometheus spec with placement smithi175=a;count:1 2024-09-16T10:33:26.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:25 smithi164 ceph-mon[25828]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:26.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:25 smithi164 ceph-mon[25828]: Saving service prometheus spec with placement smithi175=a;count:1 2024-09-16T10:33:26.206 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:25 smithi175 ceph-mon[30341]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:26.206 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:25 smithi175 ceph-mon[30341]: Saving service prometheus spec with placement smithi175=a;count:1 2024-09-16T10:33:26.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:25 smithi175 podman[67019]: 2024-09-16 10:33:25.95675037 +0000 UTC m=+0.582615940 container init 1e9a78c1fad707560118af6790e7a697b74511fcc50d2f9bd2568ba15cd46e69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T10:33:26.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:25 smithi175 podman[67019]: 2024-09-16 10:33:25.961339608 +0000 UTC m=+0.587205182 container start 1e9a78c1fad707560118af6790e7a697b74511fcc50d2f9bd2568ba15cd46e69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T10:33:26.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 bash[67019]: 1e9a78c1fad707560118af6790e7a697b74511fcc50d2f9bd2568ba15cd46e69 2024-09-16T10:33:26.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 systemd[1]: Started Ceph iscsi.iscsi.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:33:26.485 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug Started the configuration object watcher 2024-09-16T10:33:26.485 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug Checking for config object changes every 1s 2024-09-16T10:33:26.485 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug Processing osd blocklist entries for this node 2024-09-16T10:33:26.888 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug Reading the configuration object to update local LIO configuration 2024-09-16T10:33:26.889 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug Configuration does not have an entry for this host(smithi175.front.sepia.ceph.com) - nothing to define to LIO 2024-09-16T10:33:26.889 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-16T10:33:26.889 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: * Environment: production 2024-09-16T10:33:26.889 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T10:33:26.889 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: Use a production WSGI server instead. 2024-09-16T10:33:26.889 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: * Debug mode: off 2024-09-16T10:33:26.889 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug * Running on all addresses. 2024-09-16T10:33:26.889 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T10:33:26.889 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: * Running on all addresses. 2024-09-16T10:33:26.889 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T10:33:26.890 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug * Running on http://[fe80::ec4:7aff:fe88:71f1]:5000/ (Press CTRL+C to quit) 2024-09-16T10:33:26.890 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: * Running on http://[fe80::ec4:7aff:fe88:71f1]:5000/ (Press CTRL+C to quit) 2024-09-16T10:33:27.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[25828]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 5.8 KiB/s wr, 185 op/s 2024-09-16T10:33:27.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:27.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/3069252247' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T10:33:27.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[40346]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 5.8 KiB/s wr, 185 op/s 2024-09-16T10:33:27.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:27.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:26 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/3069252247' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T10:33:27.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-mon[30341]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 5.8 KiB/s wr, 185 op/s 2024-09-16T10:33:27.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:27.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:27.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:26 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/3069252247' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T10:33:28.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:27 smithi175 ceph-mon[30341]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T10:33:28.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:27 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:28.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:27 smithi164 ceph-mon[25828]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T10:33:28.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:27 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:28.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:27 smithi164 ceph-mon[40346]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T10:33:28.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:27 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:28.226 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled node-exporter update... 2024-09-16T10:33:29.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:28 smithi164 ceph-mon[25828]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 141 op/s 2024-09-16T10:33:29.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:28 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:29.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:28 smithi164 ceph-mon[25828]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T10:33:29.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:28 smithi164 ceph-mon[40346]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 141 op/s 2024-09-16T10:33:29.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:28 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:29.195 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:28 smithi164 ceph-mon[40346]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T10:33:29.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:28 smithi175 ceph-mon[30341]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 141 op/s 2024-09-16T10:33:29.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:28 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:29.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:28 smithi175 ceph-mon[30341]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T10:33:29.261 DEBUG:teuthology.orchestra.run.smithi164:node-exporter.a> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-16T10:33:29.264 DEBUG:teuthology.orchestra.run.smithi175:node-exporter.b> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-16T10:33:29.267 INFO:tasks.cephadm:Adding alertmanager.a on smithi164 2024-09-16T10:33:29.267 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi164=a' 2024-09-16T10:33:29.852 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:33:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:29 smithi175 ceph-mon[30341]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi164=a;smithi175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:29 smithi175 ceph-mon[30341]: Saving service node-exporter spec with placement smithi164=a;smithi175=b;count:2 2024-09-16T10:33:30.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:29 smithi164 ceph-mon[25828]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi164=a;smithi175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:30.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:29 smithi164 ceph-mon[25828]: Saving service node-exporter spec with placement smithi164=a;smithi175=b;count:2 2024-09-16T10:33:30.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:29 smithi164 ceph-mon[40346]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi164=a;smithi175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:30.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:29 smithi164 ceph-mon[40346]: Saving service node-exporter spec with placement smithi164=a;smithi175=b;count:2 2024-09-16T10:33:31.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[40346]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 3.8 KiB/s wr, 124 op/s 2024-09-16T10:33:31.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:31.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:31.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:31.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[25828]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 3.8 KiB/s wr, 124 op/s 2024-09-16T10:33:31.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:31.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:31.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:31.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:31.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:30 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:31.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:30 smithi175 ceph-mon[30341]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 3.8 KiB/s wr, 124 op/s 2024-09-16T10:33:31.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:30 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:31.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:30 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:31.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:30 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:31.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:30 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:31.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:30 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:32.037 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled alertmanager update... 2024-09-16T10:33:32.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:31 smithi164 ceph-mon[25828]: Deploying daemon prometheus.a on smithi175 2024-09-16T10:33:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:31 smithi164 ceph-mon[40346]: Deploying daemon prometheus.a on smithi175 2024-09-16T10:33:32.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:31 smithi175 ceph-mon[30341]: Deploying daemon prometheus.a on smithi175 2024-09-16T10:33:33.189 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:32 smithi175 ceph-mon[30341]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 3.0 KiB/s wr, 95 op/s 2024-09-16T10:33:33.190 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:33.190 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:32 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:33.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:32 smithi164 ceph-mon[40346]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 3.0 KiB/s wr, 95 op/s 2024-09-16T10:33:33.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:33.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:32 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:33.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:32 smithi164 ceph-mon[25828]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 3.0 KiB/s wr, 95 op/s 2024-09-16T10:33:33.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:33.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:32 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:33.207 DEBUG:teuthology.orchestra.run.smithi164:alertmanager.a> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-16T10:33:33.210 INFO:tasks.cephadm:Adding grafana.a on smithi175 2024-09-16T10:33:33.210 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi175=a' 2024-09-16T10:33:33.548 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:33:33.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:33 smithi175 ceph-mon[30341]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi164=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:33.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:33 smithi175 ceph-mon[30341]: Saving service alertmanager spec with placement smithi164=a;count:1 2024-09-16T10:33:34.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:33 smithi164 ceph-mon[40346]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi164=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:34.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:33 smithi164 ceph-mon[40346]: Saving service alertmanager spec with placement smithi164=a;count:1 2024-09-16T10:33:34.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:33 smithi164 ceph-mon[25828]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi164=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:34.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:33 smithi164 ceph-mon[25828]: Saving service alertmanager spec with placement smithi164=a;count:1 2024-09-16T10:33:35.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:34 smithi164 ceph-mon[40346]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 2.4 KiB/s wr, 73 op/s 2024-09-16T10:33:35.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:34 smithi164 ceph-mon[25828]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 2.4 KiB/s wr, 73 op/s 2024-09-16T10:33:35.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:34 smithi175 ceph-mon[30341]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 2.4 KiB/s wr, 73 op/s 2024-09-16T10:33:36.425 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled grafana update... 2024-09-16T10:33:36.735 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:36 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug there is no tcmu-runner data available 2024-09-16T10:33:37.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:36 smithi164 ceph-mon[40346]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.7 KiB/s wr, 49 op/s 2024-09-16T10:33:37.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:36 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:37.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:36 smithi164 ceph-mon[25828]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.7 KiB/s wr, 49 op/s 2024-09-16T10:33:37.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:36 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:37.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:36 smithi175 ceph-mon[30341]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.7 KiB/s wr, 49 op/s 2024-09-16T10:33:37.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:36 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:37.812 DEBUG:teuthology.orchestra.run.smithi175:grafana.a> sudo journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@grafana.a.service 2024-09-16T10:33:37.814 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T10:33:37.815 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T10:33:38.024 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:38.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:37 smithi164 ceph-mon[40346]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:33:38.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:37 smithi164 ceph-mon[40346]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:38.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:37 smithi164 ceph-mon[40346]: Saving service grafana spec with placement smithi175=a;count:1 2024-09-16T10:33:38.136 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:37 smithi164 ceph-mon[25828]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:33:38.136 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:37 smithi164 ceph-mon[25828]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:38.136 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:37 smithi164 ceph-mon[25828]: Saving service grafana spec with placement smithi175=a;count:1 2024-09-16T10:33:38.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:37 smithi175 ceph-mon[30341]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:33:38.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:37 smithi175 ceph-mon[30341]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:33:38.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:37 smithi175 ceph-mon[30341]: Saving service grafana spec with placement smithi175=a;count:1 2024-09-16T10:33:39.161 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:38 smithi175 ceph-mon[30341]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T10:33:39.166 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:38 smithi164 ceph-mon[40346]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T10:33:39.166 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:38 smithi164 ceph-mon[25828]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T10:33:39.877 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-09-16T10:33:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCDCehmPkIJNBAAf4PW4CxuTeoFtqmkqbUplw== 2024-09-16T10:33:40.082 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:39 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1635436117' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T10:33:40.083 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:39 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1635436117' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T10:33:40.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:39 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1635436117' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T10:33:40.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:39 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1635436117' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T10:33:40.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:39 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1635436117' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T10:33:40.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:39 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1635436117' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T10:33:40.627 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:33:40.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T10:33:40.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T10:33:40.665 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T10:33:40.963 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:40 smithi175 ceph-mon[30341]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T10:33:41.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:40 smithi164 ceph-mon[40346]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T10:33:41.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:40 smithi164 ceph-mon[25828]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-16T10:33:41.200 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T10:33:42.976 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:42 smithi175 ceph-mon[30341]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-16T10:33:42.977 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:33:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:33:42.955Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T10:33:42.977 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:33:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:33:42.975Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=19.322205ms db_storage=1.114µs remote_storage=1.392µs web_handler=424ns query_engine=1.423µs scrape=608.483µs scrape_sd=247.666µs notify=686ns notify_sd=1.382µs rules=17.842606ms tracing=5.389µs 2024-09-16T10:33:42.977 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:33:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:33:42.975Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-16T10:33:42.977 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:33:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:33:42.975Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T10:33:43.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:42 smithi164 ceph-mon[25828]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-16T10:33:43.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:42 smithi164 ceph-mon[40346]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-16T10:33:43.235 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:33:43 smithi175 bash[68863]: 49b322f3700fdc48b9d6c95a058102ab28c2dfc7e00fdcdbc631d6dc4a7fbb4d 2024-09-16T10:33:43.235 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:33:43 smithi175 systemd[1]: Started Ceph prometheus.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:33:44.423 INFO:teuthology.orchestra.run.smithi175.stdout:[client.1] 2024-09-16T10:33:44.423 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCHCehmGICNLhAA+tWdCxFrDqM+5SETMa3Fsw== 2024-09-16T10:33:44.424 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:44.424 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:44.424 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:44.424 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T10:33:44.424 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-mon[30341]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:33:44.424 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-mon[30341]: from='client.? 172.21.15.175:0/4137124761' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T10:33:44.424 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-mon[30341]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T10:33:44.424 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-mon[30341]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T10:33:44.424 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: ignoring --setuser ceph since I am not root 2024-09-16T10:33:44.425 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: ignoring --setgroup ceph since I am not root 2024-09-16T10:33:44.425 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-mgr[32427]: -- 172.21.15.175:0/4168929262 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5573d2424d00 con 0x5573d23f9400 2024-09-16T10:33:44.425 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:44.243+0000 7f949d566040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T10:33:44.425 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:44.299+0000 7f949d566040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T10:33:44.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:44.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:44.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:44.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T10:33:44.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[40346]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:33:44.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[40346]: from='client.? 172.21.15.175:0/4137124761' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T10:33:44.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[40346]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T10:33:44.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[40346]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T10:33:44.444 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ignoring --setuser ceph since I am not root 2024-09-16T10:33:44.444 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ignoring --setgroup ceph since I am not root 2024-09-16T10:33:44.445 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:44.246+0000 7fe0ac837040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T10:33:44.445 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:44.303+0000 7fe0ac837040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T10:33:44.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:44.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:44.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' 2024-09-16T10:33:44.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T10:33:44.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[25828]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:33:44.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[25828]: from='client.? 172.21.15.175:0/4137124761' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T10:33:44.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[25828]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T10:33:44.445 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-mon[25828]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T10:33:44.735 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:44.448+0000 7f949d566040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T10:33:44.735 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:44.513+0000 7f949d566040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T10:33:44.735 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:44.657+0000 7f949d566040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:33:44.802 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:44.451+0000 7fe0ac837040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T10:33:44.802 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:44.517+0000 7fe0ac837040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T10:33:44.802 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:44.661+0000 7fe0ac837040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:33:44.802 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:44.800+0000 7fe0ac837040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T10:33:45.101 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:44 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:44.798+0000 7f949d566040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T10:33:45.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:45 smithi164 ceph-mon[40346]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T10:33:45.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:45 smithi164 ceph-mon[40346]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-16T10:33:45.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:45 smithi164 ceph-mon[25828]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T10:33:45.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:45 smithi164 ceph-mon[25828]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-16T10:33:45.444 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:45 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:45.106+0000 7fe0ac837040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T10:33:45.444 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:45 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:45.166+0000 7fe0ac837040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T10:33:45.444 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:45 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:45.346+0000 7fe0ac837040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T10:33:45.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:45 smithi175 ceph-mon[30341]: from='mgr.14150 172.21.15.164:0/2181507837' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T10:33:45.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:45 smithi175 ceph-mon[30341]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-16T10:33:45.485 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:45 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:45.104+0000 7f949d566040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T10:33:45.486 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:45 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:45.166+0000 7f949d566040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T10:33:45.486 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:45 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:45.348+0000 7f949d566040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T10:33:45.599 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:33:45.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-16T10:33:45.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T10:33:45.637 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T10:33:45.637 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T10:33:45.638 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T10:33:45.847 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:46.485 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:46 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:46.218+0000 7f949d566040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T10:33:46.485 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:46 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:46.283+0000 7f949d566040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T10:33:46.694 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:46.269+0000 7fe0ac837040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T10:33:46.695 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:46.335+0000 7fe0ac837040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T10:33:47.235 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:46 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:46.982+0000 7f949d566040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T10:33:47.235 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:47.094+0000 7f949d566040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T10:33:47.316 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:47.048+0000 7fe0ac837040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T10:33:47.317 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:47.170+0000 7fe0ac837040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T10:33:47.506 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:47.234+0000 7f949d566040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:33:47.506 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:47.294+0000 7f949d566040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T10:33:47.506 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:47.437+0000 7f949d566040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T10:33:47.506 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:47.504+0000 7f949d566040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T10:33:47.581 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:47.315+0000 7fe0ac837040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T10:33:47.581 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:47.383+0000 7fe0ac837040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T10:33:47.581 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:47.534+0000 7fe0ac837040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T10:33:47.781 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:47.563+0000 7f949d566040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T10:33:47.781 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:47.716+0000 7f949d566040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T10:33:47.782 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:47.779+0000 7f949d566040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T10:33:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:33:47.832 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:47.605+0000 7fe0ac837040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T10:33:47.832 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:47.667+0000 7fe0ac837040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T10:33:47.832 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:47.830+0000 7fe0ac837040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T10:33:48.046 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/388747225' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T10:33:48.047 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:47.846+0000 7f949d566040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T10:33:48.047 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:47.929+0000 7f949d566040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T10:33:48.047 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:47.987+0000 7f949d566040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T10:33:48.047 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:48.044+0000 7f949d566040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T10:33:48.110 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/388747225' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T10:33:48.110 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/388747225' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T10:33:48.111 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:47.899+0000 7fe0ac837040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T10:33:48.111 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:47.966+0000 7fe0ac837040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T10:33:48.111 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:48.048+0000 7fe0ac837040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T10:33:48.111 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:48.108+0000 7fe0ac837040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T10:33:48.443 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:48.169+0000 7fe0ac837040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T10:33:48.514 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1282888897},{"type":"v1","addr":"172.21.15.164:6801","nonce":1282888897}]},"active_addr":"172.21.15.164:6801/1282888897","active_change":"2024-09-16T10:28:47.696629+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24115,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.164:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":4255763740}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":2129833969}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":3229272113}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":3357445173}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":4159389662}]}]} 2024-09-16T10:33:48.519 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T10:33:48.519 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T10:33:48.519 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T10:33:48.720 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:49.155 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:48.978+0000 7fe0ac837040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T10:33:49.155 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:33:49] ENGINE Bus STARTING 2024-09-16T10:33:49.155 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: CherryPy Checker: 2024-09-16T10:33:49.155 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: The Application mounted at '' has an empty config. 2024-09-16T10:33:49.156 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:49.156 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[40346]: Standby manager daemon x restarted 2024-09-16T10:33:49.156 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[40346]: Standby manager daemon x started 2024-09-16T10:33:49.156 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[40346]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T10:33:49.156 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[40346]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T10:33:49.156 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[40346]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T10:33:49.156 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[40346]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T10:33:49.157 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[25828]: Standby manager daemon x restarted 2024-09-16T10:33:49.157 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[25828]: Standby manager daemon x started 2024-09-16T10:33:49.157 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[25828]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T10:33:49.157 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[25828]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T10:33:49.157 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[25828]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T10:33:49.157 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:48 smithi164 ceph-mon[25828]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T10:33:49.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-mon[30341]: Standby manager daemon x restarted 2024-09-16T10:33:49.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-mon[30341]: Standby manager daemon x started 2024-09-16T10:33:49.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-mon[30341]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T10:33:49.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-mon[30341]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T10:33:49.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-mon[30341]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T10:33:49.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-mon[30341]: from='mgr.? 172.21.15.175:0/1954539863' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T10:33:49.236 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:48.808+0000 7f949d566040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T10:33:49.236 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: [16/Sep/2024:10:33:48] ENGINE Bus STARTING 2024-09-16T10:33:49.236 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: CherryPy Checker: 2024-09-16T10:33:49.236 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: The Application mounted at '' has an empty config. 2024-09-16T10:33:49.236 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: 2024-09-16T10:33:49.236 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: [16/Sep/2024:10:33:48] ENGINE Serving on http://:::9283 2024-09-16T10:33:49.236 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:33:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x[32404]: [16/Sep/2024:10:33:48] ENGINE Bus STARTED 2024-09-16T10:33:49.443 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:33:49] ENGINE Serving on http://:::9283 2024-09-16T10:33:49.444 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:33:49] ENGINE Bus STARTED 2024-09-16T10:33:50.109 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug there is no tcmu-runner data available 2024-09-16T10:33:50.110 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T10:33:50.110 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: Active manager daemon y restarted 2024-09-16T10:33:50.110 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: Activating manager daemon y 2024-09-16T10:33:50.110 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T10:33:50.110 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: mgrmap e19: y(active, starting, since 0.00834941s), standbys: x 2024-09-16T10:33:50.110 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:33:50.110 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:33:50.110 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:33:50.110 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: Manager daemon y is now available 2024-09-16T10:33:50.111 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:50.112 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:50.112 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:33:50.112 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T10:33:50.112 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T10:33:50.112 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T10:33:50.112 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:49 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T10:33:50.158 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T10:33:50.158 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: Active manager daemon y restarted 2024-09-16T10:33:50.158 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: Activating manager daemon y 2024-09-16T10:33:50.158 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T10:33:50.158 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: mgrmap e19: y(active, starting, since 0.00834941s), standbys: x 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:33:50.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:33:50.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T10:33:50.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T10:33:50.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T10:33:50.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: Manager daemon y is now available 2024-09-16T10:33:50.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:50.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:50.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:33:50.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T10:33:50.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T10:33:50.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T10:33:50.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: Active manager daemon y restarted 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: Activating manager daemon y 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: mgrmap e19: y(active, starting, since 0.00834941s), standbys: x 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T10:33:50.161 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: Manager daemon y is now available 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:33:50.162 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T10:33:50.163 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T10:33:50.163 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T10:33:50.163 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:49 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T10:33:50.429 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:50.341+0000 7fe00a531640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:10:33:50] ENGINE Error in HTTPServer.serve 2024-09-16T10:33:50.429 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: Traceback (most recent call last): 2024-09-16T10:33:50.429 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T10:33:50.430 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self._connections.run(self.expiration_interval) 2024-09-16T10:33:50.430 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T10:33:50.430 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self._run(expiration_interval) 2024-09-16T10:33:50.430 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T10:33:50.430 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T10:33:50.430 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T10:33:50.430 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T10:33:50.431 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T10:33:50.431 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: s = self.context.wrap_socket( 2024-09-16T10:33:50.431 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T10:33:50.431 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: return self.sslsocket_class._create( 2024-09-16T10:33:50.431 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T10:33:50.431 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self.do_handshake() 2024-09-16T10:33:50.431 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T10:33:50.431 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: self._sslobj.do_handshake() 2024-09-16T10:33:50.432 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T10:33:50.432 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:33:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:33:50.642 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":70,"fsid":"3d5087c4-7416-11ef-bceb-c7b262605968","created":"2024-09-16T10:27:53.771794+0000","modified":"2024-09-16T10:33:48.980563+0000","last_up_change":"2024-09-16T10:32:57.974199+0000","last_in_change":"2024-09-16T10:32:40.215908+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T10:30:59.996499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-16T10:33:12.433262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-16T10:33:12.665380+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T10:33:14.109898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T10:33:16.085191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T10:33:18.161892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8ae783-a95a-4a6e-ad9b-fa68ce6256a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6803","nonce":732894384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6805","nonce":732894384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6809","nonce":732894384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6807","nonce":732894384}]},"public_addr":"172.21.15.164:6803/732894384","cluster_addr":"172.21.15.164:6805/732894384","heartbeat_back_addr":"172.21.15.164:6809/732894384","heartbeat_front_addr":"172.21.15.164:6807/732894384","state":["exists","up"]},{"osd":1,"uuid":"8da925d4-94e1-4f33-9c3f-90d6ea8574ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6811","nonce":4121909442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6813","nonce":4121909442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6817","nonce":4121909442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6815","nonce":4121909442}]},"public_addr":"172.21.15.164:6811/4121909442","cluster_addr":"172.21.15.164:6813/4121909442","heartbeat_back_addr":"172.21.15.164:6817/4121909442","heartbeat_front_addr":"172.21.15.164:6815/4121909442","state":["exists","up"]},{"osd":2,"uuid":"c8c3465b-81e1-4ef4-8574-049b2b8e7455","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6819","nonce":254163173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6821","nonce":254163173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6825","nonce":254163173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6823","nonce":254163173}]},"public_addr":"172.21.15.164:6819/254163173","cluster_addr":"172.21.15.164:6821/254163173","heartbeat_back_addr":"172.21.15.164:6825/254163173","heartbeat_front_addr":"172.21.15.164:6823/254163173","state":["exists","up"]},{"osd":3,"uuid":"0662cc54-7dac-48dd-8802-8f82c6424867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6827","nonce":3793297178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6829","nonce":3793297178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6833","nonce":3793297178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6831","nonce":3793297178}]},"public_addr":"172.21.15.164:6827/3793297178","cluster_addr":"172.21.15.164:6829/3793297178","heartbeat_back_addr":"172.21.15.164:6833/3793297178","heartbeat_front_addr":"172.21.15.164:6831/3793297178","state":["exists","up"]},{"osd":4,"uuid":"c0745f5b-704f-420b-82de-12d555db0e27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6801","nonce":2959667189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6803","nonce":2959667189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6807","nonce":2959667189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6805","nonce":2959667189}]},"public_addr":"172.21.15.175:6801/2959667189","cluster_addr":"172.21.15.175:6803/2959667189","heartbeat_back_addr":"172.21.15.175:6807/2959667189","heartbeat_front_addr":"172.21.15.175:6805/2959667189","state":["exists","up"]},{"osd":5,"uuid":"136e5111-e72a-4acf-aa93-63fe31c75284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6809","nonce":1222908899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6811","nonce":1222908899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6815","nonce":1222908899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6813","nonce":1222908899}]},"public_addr":"172.21.15.175:6809/1222908899","cluster_addr":"172.21.15.175:6811/1222908899","heartbeat_back_addr":"172.21.15.175:6815/1222908899","heartbeat_front_addr":"172.21.15.175:6813/1222908899","state":["exists","up"]},{"osd":6,"uuid":"766b9300-83a6-4ab4-a0d2-b7cb9e32ed60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6817","nonce":2771950113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6819","nonce":2771950113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6823","nonce":2771950113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6821","nonce":2771950113}]},"public_addr":"172.21.15.175:6817/2771950113","cluster_addr":"172.21.15.175:6819/2771950113","heartbeat_back_addr":"172.21.15.175:6823/2771950113","heartbeat_front_addr":"172.21.15.175:6821/2771950113","state":["exists","up"]},{"osd":7,"uuid":"0a9dadc7-445b-4e0c-b8a5-1003634eb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6825","nonce":1006272747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6827","nonce":1006272747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6831","nonce":1006272747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6829","nonce":1006272747}]},"public_addr":"172.21.15.175:6825/1006272747","cluster_addr":"172.21.15.175:6827/1006272747","heartbeat_back_addr":"172.21.15.175:6831/1006272747","heartbeat_front_addr":"172.21.15.175:6829/1006272747","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:30:08.753272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:30:32.724150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:30:56.799127+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:31:21.216920+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:31:43.058730+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:32:07.087680+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:32:31.368288+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:32:56.035998+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/4159389662":"2024-09-17T10:33:48.980532+0000","172.21.15.164:6800/1282888897":"2024-09-17T10:33:48.980532+0000","172.21.15.164:0/3711213828":"2024-09-17T10:28:24.597753+0000","172.21.15.164:0/2129833969":"2024-09-17T10:33:48.980532+0000","172.21.15.164:0/3110320769":"2024-09-17T10:28:24.597753+0000","172.21.15.164:6800/3165220513":"2024-09-17T10:28:47.696520+0000","172.21.15.164:0/3996721572":"2024-09-17T10:28:24.597753+0000","172.21.15.164:0/3357445173":"2024-09-17T10:33:48.980532+0000","172.21.15.164:0/4255763740":"2024-09-17T10:33:48.980532+0000","172.21.15.164:6800/2694972888":"2024-09-17T10:28:24.597753+0000","172.21.15.164:6801/2694972888":"2024-09-17T10:28:24.597753+0000","172.21.15.164:0/430719755":"2024-09-17T10:28:47.696520+0000","172.21.15.164:6801/1282888897":"2024-09-17T10:33:48.980532+0000","172.21.15.164:0/1934115177":"2024-09-17T10:28:47.696520+0000","172.21.15.164:0/3229272113":"2024-09-17T10:33:48.980532+0000","172.21.15.164:6801/3165220513":"2024-09-17T10:28:47.696520+0000","172.21.15.164:0/699939050":"2024-09-17T10:28:47.696520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T10:33:51.259 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:50 smithi175 ceph-mon[30341]: mgrmap e20: y(active, since 1.01016s), standbys: x 2024-09-16T10:33:51.464 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:50 smithi175 ceph-mon[30341]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:33:51.464 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:50 smithi175 ceph-mon[30341]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:51.465 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:50 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/3189637088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T10:33:51.465 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-mon[40346]: mgrmap e20: y(active, since 1.01016s), standbys: x 2024-09-16T10:33:51.465 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-mon[40346]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:33:51.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-mon[40346]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:51.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/3189637088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T10:33:51.466 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-mon[25828]: mgrmap e20: y(active, since 1.01016s), standbys: x 2024-09-16T10:33:51.466 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-mon[25828]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:33:51.466 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-mon[25828]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:51.466 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:50 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3189637088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T10:33:51.654 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T10:33:51.655 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T10:33:52.185 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:51 smithi175 ceph-mon[30341]: [16/Sep/2024:10:33:50] ENGINE Bus STARTING 2024-09-16T10:33:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:51 smithi175 ceph-mon[30341]: [16/Sep/2024:10:33:50] ENGINE Serving on https://172.21.15.164:7150 2024-09-16T10:33:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:51 smithi175 ceph-mon[30341]: [16/Sep/2024:10:33:50] ENGINE Error in HTTPServer.serve 2024-09-16T10:33:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout: Traceback (most recent call last): 2024-09-16T10:33:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T10:33:52.235 INFO:journalctl@ceph.mon.b.smithi175.stdout: self._connections.run(self.expiration_interval) 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: self._run(expiration_interval) 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: s = self.context.wrap_socket( 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: return self.sslsocket_class._create( 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: self.do_handshake() 2024-09-16T10:33:52.236 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T10:33:52.237 INFO:journalctl@ceph.mon.b.smithi175.stdout: self._sslobj.do_handshake() 2024-09-16T10:33:52.237 INFO:journalctl@ceph.mon.b.smithi175.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T10:33:52.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:51 smithi175 ceph-mon[30341]: [16/Sep/2024:10:33:50] ENGINE Serving on http://172.21.15.164:8765 2024-09-16T10:33:52.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:51 smithi175 ceph-mon[30341]: [16/Sep/2024:10:33:50] ENGINE Bus STARTED 2024-09-16T10:33:52.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:51 smithi175 ceph-mon[30341]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:52.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:51 smithi175 ceph-mon[30341]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[40346]: [16/Sep/2024:10:33:50] ENGINE Bus STARTING 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[40346]: [16/Sep/2024:10:33:50] ENGINE Serving on https://172.21.15.164:7150 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[40346]: [16/Sep/2024:10:33:50] ENGINE Error in HTTPServer.serve 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout: Traceback (most recent call last): 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout: self._connections.run(self.expiration_interval) 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout: self._run(expiration_interval) 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T10:33:52.307 INFO:journalctl@ceph.mon.c.smithi164.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout: s = self.context.wrap_socket( 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout: return self.sslsocket_class._create( 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout: self.do_handshake() 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout: self._sslobj.do_handshake() 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[40346]: [16/Sep/2024:10:33:50] ENGINE Serving on http://172.21.15.164:8765 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[40346]: [16/Sep/2024:10:33:50] ENGINE Bus STARTED 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[40346]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:52.308 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[40346]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[25828]: [16/Sep/2024:10:33:50] ENGINE Bus STARTING 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[25828]: [16/Sep/2024:10:33:50] ENGINE Serving on https://172.21.15.164:7150 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[25828]: [16/Sep/2024:10:33:50] ENGINE Error in HTTPServer.serve 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout: Traceback (most recent call last): 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._connections.run(self.expiration_interval) 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._run(expiration_interval) 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T10:33:52.309 INFO:journalctl@ceph.mon.a.smithi164.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout: s = self.context.wrap_socket( 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout: return self.sslsocket_class._create( 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout: self.do_handshake() 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._sslobj.do_handshake() 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[25828]: [16/Sep/2024:10:33:50] ENGINE Serving on http://172.21.15.164:8765 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[25828]: [16/Sep/2024:10:33:50] ENGINE Bus STARTED 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[25828]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:52.310 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:51 smithi164 ceph-mon[25828]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T10:33:53.588 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.588 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.588 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.589 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.589 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.589 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.589 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.589 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.590 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.590 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.590 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.590 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.590 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: Adjusting osd_memory_target on smithi175 to 3760M 2024-09-16T10:33:53.590 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.591 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[40346]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:53.591 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.591 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.591 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.591 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.592 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.592 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.592 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.592 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.592 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.593 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.593 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.593 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.593 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: Adjusting osd_memory_target on smithi175 to 3760M 2024-09-16T10:33:53.593 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.593 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:53 smithi164 ceph-mon[25828]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.737 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.737 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.737 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:53.739 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: Adjusting osd_memory_target on smithi175 to 3760M 2024-09-16T10:33:53.739 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:53.739 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:53 smithi175 ceph-mon[30341]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:53.971 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:33:53.971 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":70,"fsid":"3d5087c4-7416-11ef-bceb-c7b262605968","created":"2024-09-16T10:27:53.771794+0000","modified":"2024-09-16T10:33:48.980563+0000","last_up_change":"2024-09-16T10:32:57.974199+0000","last_in_change":"2024-09-16T10:32:40.215908+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T10:30:59.996499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-16T10:33:12.433262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-16T10:33:12.665380+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T10:33:14.109898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T10:33:16.085191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T10:33:18.161892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8ae783-a95a-4a6e-ad9b-fa68ce6256a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6803","nonce":732894384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6805","nonce":732894384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6809","nonce":732894384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":732894384},{"type":"v1","addr":"172.21.15.164:6807","nonce":732894384}]},"public_addr":"172.21.15.164:6803/732894384","cluster_addr":"172.21.15.164:6805/732894384","heartbeat_back_addr":"172.21.15.164:6809/732894384","heartbeat_front_addr":"172.21.15.164:6807/732894384","state":["exists","up"]},{"osd":1,"uuid":"8da925d4-94e1-4f33-9c3f-90d6ea8574ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6811","nonce":4121909442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6813","nonce":4121909442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6817","nonce":4121909442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":4121909442},{"type":"v1","addr":"172.21.15.164:6815","nonce":4121909442}]},"public_addr":"172.21.15.164:6811/4121909442","cluster_addr":"172.21.15.164:6813/4121909442","heartbeat_back_addr":"172.21.15.164:6817/4121909442","heartbeat_front_addr":"172.21.15.164:6815/4121909442","state":["exists","up"]},{"osd":2,"uuid":"c8c3465b-81e1-4ef4-8574-049b2b8e7455","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6819","nonce":254163173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6821","nonce":254163173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6825","nonce":254163173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":254163173},{"type":"v1","addr":"172.21.15.164:6823","nonce":254163173}]},"public_addr":"172.21.15.164:6819/254163173","cluster_addr":"172.21.15.164:6821/254163173","heartbeat_back_addr":"172.21.15.164:6825/254163173","heartbeat_front_addr":"172.21.15.164:6823/254163173","state":["exists","up"]},{"osd":3,"uuid":"0662cc54-7dac-48dd-8802-8f82c6424867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6827","nonce":3793297178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6829","nonce":3793297178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6833","nonce":3793297178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3793297178},{"type":"v1","addr":"172.21.15.164:6831","nonce":3793297178}]},"public_addr":"172.21.15.164:6827/3793297178","cluster_addr":"172.21.15.164:6829/3793297178","heartbeat_back_addr":"172.21.15.164:6833/3793297178","heartbeat_front_addr":"172.21.15.164:6831/3793297178","state":["exists","up"]},{"osd":4,"uuid":"c0745f5b-704f-420b-82de-12d555db0e27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6801","nonce":2959667189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6803","nonce":2959667189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6807","nonce":2959667189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2959667189},{"type":"v1","addr":"172.21.15.175:6805","nonce":2959667189}]},"public_addr":"172.21.15.175:6801/2959667189","cluster_addr":"172.21.15.175:6803/2959667189","heartbeat_back_addr":"172.21.15.175:6807/2959667189","heartbeat_front_addr":"172.21.15.175:6805/2959667189","state":["exists","up"]},{"osd":5,"uuid":"136e5111-e72a-4acf-aa93-63fe31c75284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6809","nonce":1222908899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6811","nonce":1222908899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6815","nonce":1222908899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1222908899},{"type":"v1","addr":"172.21.15.175:6813","nonce":1222908899}]},"public_addr":"172.21.15.175:6809/1222908899","cluster_addr":"172.21.15.175:6811/1222908899","heartbeat_back_addr":"172.21.15.175:6815/1222908899","heartbeat_front_addr":"172.21.15.175:6813/1222908899","state":["exists","up"]},{"osd":6,"uuid":"766b9300-83a6-4ab4-a0d2-b7cb9e32ed60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6817","nonce":2771950113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6819","nonce":2771950113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6823","nonce":2771950113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2771950113},{"type":"v1","addr":"172.21.15.175:6821","nonce":2771950113}]},"public_addr":"172.21.15.175:6817/2771950113","cluster_addr":"172.21.15.175:6819/2771950113","heartbeat_back_addr":"172.21.15.175:6823/2771950113","heartbeat_front_addr":"172.21.15.175:6821/2771950113","state":["exists","up"]},{"osd":7,"uuid":"0a9dadc7-445b-4e0c-b8a5-1003634eb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6825","nonce":1006272747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6827","nonce":1006272747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6831","nonce":1006272747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":1006272747},{"type":"v1","addr":"172.21.15.175:6829","nonce":1006272747}]},"public_addr":"172.21.15.175:6825/1006272747","cluster_addr":"172.21.15.175:6827/1006272747","heartbeat_back_addr":"172.21.15.175:6831/1006272747","heartbeat_front_addr":"172.21.15.175:6829/1006272747","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:30:08.753272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:30:32.724150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:30:56.799127+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:31:21.216920+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:31:43.058730+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:32:07.087680+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:32:31.368288+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T10:32:56.035998+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/4159389662":"2024-09-17T10:33:48.980532+0000","172.21.15.164:6800/1282888897":"2024-09-17T10:33:48.980532+0000","172.21.15.164:0/3711213828":"2024-09-17T10:28:24.597753+0000","172.21.15.164:0/2129833969":"2024-09-17T10:33:48.980532+0000","172.21.15.164:0/3110320769":"2024-09-17T10:28:24.597753+0000","172.21.15.164:6800/3165220513":"2024-09-17T10:28:47.696520+0000","172.21.15.164:0/3996721572":"2024-09-17T10:28:24.597753+0000","172.21.15.164:0/3357445173":"2024-09-17T10:33:48.980532+0000","172.21.15.164:0/4255763740":"2024-09-17T10:33:48.980532+0000","172.21.15.164:6800/2694972888":"2024-09-17T10:28:24.597753+0000","172.21.15.164:6801/2694972888":"2024-09-17T10:28:24.597753+0000","172.21.15.164:0/430719755":"2024-09-17T10:28:47.696520+0000","172.21.15.164:6801/1282888897":"2024-09-17T10:33:48.980532+0000","172.21.15.164:0/1934115177":"2024-09-17T10:28:47.696520+0000","172.21.15.164:0/3229272113":"2024-09-17T10:33:48.980532+0000","172.21.15.164:6801/3165220513":"2024-09-17T10:28:47.696520+0000","172.21.15.164:0/699939050":"2024-09-17T10:28:47.696520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T10:33:54.268 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ::ffff:172.21.15.175 - - [16/Sep/2024:10:33:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T10:33:54.665 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T10:33:54.665 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T10:33:54.666 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T10:33:54.666 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T10:33:54.666 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T10:33:54.667 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T10:33:54.667 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T10:33:54.668 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T10:33:54.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[40346]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T10:33:54.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:54.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:54.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1959704185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T10:33:54.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:54.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:54.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[25828]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T10:33:54.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:54.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:54.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1959704185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T10:33:54.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:54.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:54 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:55.154 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:55.154 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:55.155 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:55.155 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:55.160 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:55.226 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:33:55.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:54 smithi175 ceph-mon[30341]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T10:33:55.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:54 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:55.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:54 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:55.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:54 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1959704185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T10:33:55.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:54 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:55.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:54 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:55.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: Adjusting osd_memory_target on smithi164 to 3760M 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T10:33:55.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-16T10:33:55.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:55.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:33:55.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:55.945 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:55.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.945 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:55.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: Adjusting osd_memory_target on smithi164 to 3760M 2024-09-16T10:33:55.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:55.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:55.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:55.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T10:33:55.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-16T10:33:55.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:55.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:33:55.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:55.946 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:55 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:56.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:56.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: Adjusting osd_memory_target on smithi164 to 3760M 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-16T10:33:56.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-16T10:33:56.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-16T10:33:56.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:33:56.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:56.237 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:55 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:57.091 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[40346]: Updating smithi164:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:33:57.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[40346]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:33:57.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[40346]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:33:57.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[40346]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T10:33:57.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[40346]: Updating smithi164:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T10:33:57.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:57.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:57.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:57.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[40346]: Deploying daemon alertmanager.a on smithi164 2024-09-16T10:33:57.092 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[25828]: Updating smithi164:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:33:57.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[25828]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:33:57.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[25828]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:33:57.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[25828]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T10:33:57.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[25828]: Updating smithi164:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T10:33:57.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:57.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:57.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:57.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:56 smithi164 ceph-mon[25828]: Deploying daemon alertmanager.a on smithi164 2024-09-16T10:33:57.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:56 smithi175 ceph-mon[30341]: Updating smithi164:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T10:33:57.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:56 smithi175 ceph-mon[30341]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:33:57.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:56 smithi175 ceph-mon[30341]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-16T10:33:57.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:56 smithi175 ceph-mon[30341]: Updating smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T10:33:57.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:56 smithi175 ceph-mon[30341]: Updating smithi164:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T10:33:57.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:56 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:57.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:56 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:57.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:56 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:33:57.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:56 smithi175 ceph-mon[30341]: Deploying daemon alertmanager.a on smithi164 2024-09-16T10:33:58.183 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:33:57 smithi164 ceph-mon[40346]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-16T10:33:58.183 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:33:57 smithi164 ceph-mon[25828]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-16T10:33:58.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:33:57 smithi175 ceph-mon[30341]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-16T10:34:00.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:00 smithi164 ceph-mon[25828]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-16T10:34:00.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:00 smithi164 ceph-mon[40346]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-16T10:34:00.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:00 smithi175 ceph-mon[30341]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-16T10:34:00.485 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:33:59 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug there is no tcmu-runner data available 2024-09-16T10:34:01.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:01 smithi164 ceph-mon[40346]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:01.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:01 smithi164 ceph-mon[25828]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:01.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:01 smithi175 ceph-mon[30341]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:02.084 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:02 smithi164 ceph-mon[40346]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T10:34:02.084 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:02 smithi164 ceph-mon[25828]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T10:34:02.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:02 smithi175 ceph-mon[30341]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T10:34:04.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:04 smithi164 ceph-mon[40346]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T10:34:04.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:04 smithi164 ceph-mon[25828]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T10:34:04.194 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:04 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ::ffff:172.21.15.175 - - [16/Sep/2024:10:34:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T10:34:04.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:04 smithi175 ceph-mon[30341]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T10:34:05.415 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:05 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:05.415 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:05 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:05.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:05 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:06.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:06 smithi164 ceph-mon[40346]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T10:34:06.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:06 smithi164 ceph-mon[25828]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T10:34:06.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:06 smithi175 ceph-mon[30341]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T10:34:06.566 INFO:teuthology.orchestra.run.smithi164.stdout:77309411368 2024-09-16T10:34:06.567 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T10:34:06.748 INFO:teuthology.orchestra.run.smithi164.stdout:111669149730 2024-09-16T10:34:06.748 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T10:34:08.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:08 smithi175 ceph-mon[30341]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T10:34:08.344 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:08 smithi164 ceph-mon[25828]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T10:34:08.345 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:08 smithi164 ceph-mon[40346]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T10:34:10.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:10 smithi164 ceph-mon[25828]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:10.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:10 smithi164 ceph-mon[40346]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:10.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:10 smithi175 ceph-mon[30341]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:10.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:34:10 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug there is no tcmu-runner data available 2024-09-16T10:34:10.524 INFO:teuthology.orchestra.run.smithi164.stdout:236223201297 2024-09-16T10:34:10.525 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T10:34:11.392 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:11 smithi164 ceph-mon[25828]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:11.393 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:11 smithi164 ceph-mon[40346]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:11.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:11 smithi175 ceph-mon[30341]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:12.382 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:12 smithi164 ceph-mon[40346]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:12.382 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:12 smithi164 ceph-mon[25828]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:12.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:12 smithi175 ceph-mon[30341]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:13.371 INFO:teuthology.orchestra.run.smithi164.stdout:201863462935 2024-09-16T10:34:13.371 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T10:34:13.770 INFO:teuthology.orchestra.run.smithi164.stdout:133143986207 2024-09-16T10:34:13.770 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T10:34:14.193 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:14 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ::ffff:172.21.15.175 - - [16/Sep/2024:10:34:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T10:34:14.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:14 smithi164 ceph-mon[25828]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:14.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:14 smithi164 ceph-mon[40346]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:14.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:14 smithi175 ceph-mon[30341]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:15.135 INFO:teuthology.orchestra.run.smithi164.stdout:55834574893 2024-09-16T10:34:15.135 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T10:34:15.762 INFO:teuthology.orchestra.run.smithi164.stdout:34359738418 2024-09-16T10:34:15.763 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T10:34:16.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:16 smithi164 ceph-mon[40346]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:16.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:16 smithi164 ceph-mon[25828]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:16.233 INFO:teuthology.orchestra.run.smithi164.stdout:167503724571 2024-09-16T10:34:16.233 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T10:34:16.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:16 smithi175 ceph-mon[30341]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:16.829 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:16.866 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:17.227 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:17.228 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:18.142 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:18 smithi164 ceph-mon[25828]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:18.143 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:18 smithi164 ceph-mon[40346]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:18.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:18 smithi175 ceph-mon[30341]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:20.189 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:20 smithi164 ceph-mon[40346]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:20.189 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:20 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:20.189 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:20 smithi164 ceph-mon[25828]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:20.189 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:20 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:20.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:20 smithi175 ceph-mon[30341]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:20.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:20 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:20.485 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:34:20 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug there is no tcmu-runner data available 2024-09-16T10:34:21.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:21 smithi164 ceph-mon[40346]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:21.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:21 smithi164 ceph-mon[25828]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:21.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:21 smithi175 ceph-mon[30341]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:22.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:22 smithi164 ceph-mon[25828]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-16T10:34:22.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:22 smithi164 ceph-mon[40346]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-16T10:34:22.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:22 smithi175 ceph-mon[30341]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-16T10:34:23.766 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:23.942 INFO:teuthology.orchestra.run.smithi164.stdout:77309411372 2024-09-16T10:34:24.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:24 smithi164 ceph-mon[40346]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:24.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:24 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/153443775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T10:34:24.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:24 smithi164 ceph-mon[25828]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:24.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:24 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/153443775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T10:34:24.194 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:24 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ::ffff:172.21.15.175 - - [16/Sep/2024:10:34:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T10:34:24.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:24 smithi175 ceph-mon[30341]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:24.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:24 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/153443775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T10:34:24.609 INFO:teuthology.orchestra.run.smithi164.stdout:55834574896 2024-09-16T10:34:25.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:25 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1367135558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T10:34:25.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:25 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1367135558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T10:34:25.252 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574893 got 55834574896 for osd.1 2024-09-16T10:34:25.252 DEBUG:teuthology.parallel:result is None 2024-09-16T10:34:25.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:25 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1367135558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T10:34:26.260 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411372 for osd.2 2024-09-16T10:34:26.260 DEBUG:teuthology.parallel:result is None 2024-09-16T10:34:26.371 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:26 smithi164 ceph-mon[25828]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:26.372 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:26 smithi164 ceph-mon[40346]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:26.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:26 smithi175 ceph-mon[30341]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:28.185 INFO:teuthology.orchestra.run.smithi164.stdout:34359738421 2024-09-16T10:34:28.205 INFO:teuthology.orchestra.run.smithi164.stdout:201863462938 2024-09-16T10:34:28.396 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:28 smithi164 ceph-mon[40346]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:28.396 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:28 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1407742817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T10:34:28.396 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:28 smithi164 ceph-mon[25828]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:28.396 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:28 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1407742817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T10:34:28.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:28 smithi175 ceph-mon[30341]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:28.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:28 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1407742817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T10:34:28.938 INFO:teuthology.orchestra.run.smithi164.stdout:236223201301 2024-09-16T10:34:28.938 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:28 smithi164 podman[83171]: 2024-09-16 10:34:28.92144242 +0000 UTC m=+2.663577727 volume create feb913328b4f0d23752e3e65e956bbe969cb0da8ee2a16d026390ff6ce95f8e9 2024-09-16T10:34:29.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:29 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/452297112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T10:34:29.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:29 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/518385339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T10:34:29.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:29 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/452297112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T10:34:29.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:29 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/518385339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T10:34:29.194 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:29 smithi164 podman[83171]: 2024-09-16 10:34:29.013716101 +0000 UTC m=+2.755851405 container create d1a5e1bf4e157459e805f3182edb6df132ecf12f77a1b7306f0dce562d27c061 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:34:29.467 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:29 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/452297112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T10:34:29.468 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:29 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/518385339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T10:34:29.717 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738421 for osd.0 2024-09-16T10:34:29.717 DEBUG:teuthology.parallel:result is None 2024-09-16T10:34:30.011 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462935 got 201863462938 for osd.6 2024-09-16T10:34:30.011 DEBUG:teuthology.parallel:result is None 2024-09-16T10:34:30.244 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:30 smithi164 ceph-mon[40346]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:30.244 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:30 smithi164 ceph-mon[25828]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:30.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:30 smithi175 ceph-mon[30341]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:30.485 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:34:30 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug there is no tcmu-runner data available 2024-09-16T10:34:30.492 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 podman[83171]: 2024-09-16 10:34:30.240051598 +0000 UTC m=+3.982186901 container init d1a5e1bf4e157459e805f3182edb6df132ecf12f77a1b7306f0dce562d27c061 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:34:30.492 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 podman[83171]: 2024-09-16 10:34:30.245760247 +0000 UTC m=+3.987895552 container start d1a5e1bf4e157459e805f3182edb6df132ecf12f77a1b7306f0dce562d27c061 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:34:30.492 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[83411]: ts=2024-09-16T10:34:30.258Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-16T10:34:30.492 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[83411]: ts=2024-09-16T10:34:30.258Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-16T10:34:30.492 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[83411]: ts=2024-09-16T10:34:30.259Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.164 port=9094 2024-09-16T10:34:30.492 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[83411]: ts=2024-09-16T10:34:30.260Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-16T10:34:30.492 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[83411]: ts=2024-09-16T10:34:30.283Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T10:34:30.492 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[83411]: ts=2024-09-16T10:34:30.283Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T10:34:30.493 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[83411]: ts=2024-09-16T10:34:30.286Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-16T10:34:30.493 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[83411]: ts=2024-09-16T10:34:30.286Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-16T10:34:30.493 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 bash[83171]: d1a5e1bf4e157459e805f3182edb6df132ecf12f77a1b7306f0dce562d27c061 2024-09-16T10:34:30.599 INFO:teuthology.orchestra.run.smithi164.stdout:111669149735 2024-09-16T10:34:30.928 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:30 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:34:30] ENGINE Bus STOPPING 2024-09-16T10:34:30.928 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:30 smithi164 systemd[1]: Started Ceph alertmanager.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:34:31.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[40346]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:31.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/911816268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T10:34:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T10:34:31.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.194 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:34:31] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T10:34:31.194 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:34:31] ENGINE Bus STOPPED 2024-09-16T10:34:31.194 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:34:31] ENGINE Bus STARTING 2024-09-16T10:34:31.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[25828]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:31.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/911816268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T10:34:31.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T10:34:31.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.458 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:34:31] ENGINE Serving on http://:::9283 2024-09-16T10:34:31.458 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:31 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:34:31] ENGINE Bus STARTED 2024-09-16T10:34:31.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:31 smithi175 ceph-mon[30341]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:31.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:31 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:31 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:31 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:31 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:31 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/911816268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T10:34:31.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:31 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:31 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:31 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T10:34:31.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:31 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:31.716 INFO:teuthology.orchestra.run.smithi164.stdout:133143986211 2024-09-16T10:34:31.931 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149730 got 111669149735 for osd.3 2024-09-16T10:34:31.931 DEBUG:teuthology.parallel:result is None 2024-09-16T10:34:32.382 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:32 smithi164 ceph-mon[40346]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T10:34:32.382 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:32 smithi164 ceph-mon[40346]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T10:34:32.382 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:32 smithi164 ceph-mon[40346]: Deploying daemon grafana.a on smithi175 2024-09-16T10:34:32.382 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:32 smithi164 ceph-mon[40346]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:32.382 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:32 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/2075566510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T10:34:32.383 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:32 smithi164 ceph-mon[25828]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T10:34:32.383 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:32 smithi164 ceph-mon[25828]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T10:34:32.383 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:32 smithi164 ceph-mon[25828]: Deploying daemon grafana.a on smithi175 2024-09-16T10:34:32.383 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:32 smithi164 ceph-mon[25828]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:32.383 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:32 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/2075566510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T10:34:32.383 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:32 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[83411]: ts=2024-09-16T10:34:32.260Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000229895s 2024-09-16T10:34:32.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:32 smithi175 ceph-mon[30341]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T10:34:32.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:32 smithi175 ceph-mon[30341]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T10:34:32.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:32 smithi175 ceph-mon[30341]: Deploying daemon grafana.a on smithi175 2024-09-16T10:34:32.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:32 smithi175 ceph-mon[30341]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:32.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:32 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/2075566510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T10:34:32.516 INFO:teuthology.orchestra.run.smithi164.stdout:167503724575 2024-09-16T10:34:32.888 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201297 got 236223201301 for osd.7 2024-09-16T10:34:32.889 DEBUG:teuthology.parallel:result is None 2024-09-16T10:34:33.237 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986207 got 133143986211 for osd.4 2024-09-16T10:34:33.237 DEBUG:teuthology.parallel:result is None 2024-09-16T10:34:33.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:33 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/1656387372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T10:34:33.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:33 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/1656387372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T10:34:33.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:33 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/1656387372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T10:34:33.817 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724571 got 167503724575 for osd.5 2024-09-16T10:34:33.817 DEBUG:teuthology.parallel:result is None 2024-09-16T10:34:33.817 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T10:34:33.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T10:34:34.025 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:34.427 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:34 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ::ffff:172.21.15.175 - - [16/Sep/2024:10:34:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T10:34:34.427 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:34 smithi164 ceph-mon[25828]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:34.427 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:34 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:34.427 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:34 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:34.427 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:34 smithi164 ceph-mon[40346]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:34.428 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:34 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:34.428 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:34 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:34.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:34 smithi175 ceph-mon[30341]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:34.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:34 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:34.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:34 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:34:35.942 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-16T10:34:36.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:36 smithi164 ceph-mon[40346]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:36.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:36 smithi164 ceph-mon[25828]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:36.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:36 smithi175 ceph-mon[30341]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:36.595 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-16T10:34:35.004971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1026,"num_read_kb":869,"num_write":675,"num_write_kb":4025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":657,"ondisk_log_size":657,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":236252,"kb_used_data":7816,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":749692196,"statfs":{"total":767926730752,"available":767684808704,"internally_reserved":0,"allocated":8003584,"data_stored":4332999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":233819704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002807"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121395+0000","last_change":"2024-09-16T10:33:19.909495+0000","last_active":"2024-09-16T10:33:24.121395+0000","last_peered":"2024-09-16T10:33:24.121395+0000","last_clean":"2024-09-16T10:33:24.121395+0000","last_became_active":"2024-09-16T10:33:19.909182+0000","last_became_peered":"2024-09-16T10:33:19.909182+0000","last_unstale":"2024-09-16T10:33:24.121395+0000","last_undegraded":"2024-09-16T10:33:24.121395+0000","last_fullsized":"2024-09-16T10:33:24.121395+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:58.791582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405015+0000","last_change":"2024-09-16T10:33:13.833129+0000","last_active":"2024-09-16T10:33:24.405015+0000","last_peered":"2024-09-16T10:33:24.405015+0000","last_clean":"2024-09-16T10:33:24.405015+0000","last_became_active":"2024-09-16T10:33:13.832647+0000","last_became_peered":"2024-09-16T10:33:13.832647+0000","last_unstale":"2024-09-16T10:33:24.405015+0000","last_undegraded":"2024-09-16T10:33:24.405015+0000","last_fullsized":"2024-09-16T10:33:24.405015+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:37.041695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121171+0000","last_change":"2024-09-16T10:33:15.846271+0000","last_active":"2024-09-16T10:33:24.121171+0000","last_peered":"2024-09-16T10:33:24.121171+0000","last_clean":"2024-09-16T10:33:24.121171+0000","last_became_active":"2024-09-16T10:33:15.845997+0000","last_became_peered":"2024-09-16T10:33:15.845997+0000","last_unstale":"2024-09-16T10:33:24.121171+0000","last_undegraded":"2024-09-16T10:33:24.121171+0000","last_fullsized":"2024-09-16T10:33:24.121171+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:18:51.246178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403491+0000","last_change":"2024-09-16T10:33:17.861562+0000","last_active":"2024-09-16T10:33:24.403491+0000","last_peered":"2024-09-16T10:33:24.403491+0000","last_clean":"2024-09-16T10:33:24.403491+0000","last_became_active":"2024-09-16T10:33:17.861165+0000","last_became_peered":"2024-09-16T10:33:17.861165+0000","last_unstale":"2024-09-16T10:33:24.403491+0000","last_undegraded":"2024-09-16T10:33:24.403491+0000","last_fullsized":"2024-09-16T10:33:24.403491+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:18.765194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122371+0000","last_change":"2024-09-16T10:33:13.830466+0000","last_active":"2024-09-16T10:33:24.122371+0000","last_peered":"2024-09-16T10:33:24.122371+0000","last_clean":"2024-09-16T10:33:24.122371+0000","last_became_active":"2024-09-16T10:33:13.830209+0000","last_became_peered":"2024-09-16T10:33:13.830209+0000","last_unstale":"2024-09-16T10:33:24.122371+0000","last_undegraded":"2024-09-16T10:33:24.122371+0000","last_fullsized":"2024-09-16T10:33:24.122371+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:51.270208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403574+0000","last_change":"2024-09-16T10:33:15.848110+0000","last_active":"2024-09-16T10:33:24.403574+0000","last_peered":"2024-09-16T10:33:24.403574+0000","last_clean":"2024-09-16T10:33:24.403574+0000","last_became_active":"2024-09-16T10:33:15.847813+0000","last_became_peered":"2024-09-16T10:33:15.847813+0000","last_unstale":"2024-09-16T10:33:24.403574+0000","last_undegraded":"2024-09-16T10:33:24.403574+0000","last_fullsized":"2024-09-16T10:33:24.403574+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:27.473578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120565+0000","last_change":"2024-09-16T10:33:17.858479+0000","last_active":"2024-09-16T10:33:24.120565+0000","last_peered":"2024-09-16T10:33:24.120565+0000","last_clean":"2024-09-16T10:33:24.120565+0000","last_became_active":"2024-09-16T10:33:17.858183+0000","last_became_peered":"2024-09-16T10:33:17.858183+0000","last_unstale":"2024-09-16T10:33:24.120565+0000","last_undegraded":"2024-09-16T10:33:24.120565+0000","last_fullsized":"2024-09-16T10:33:24.120565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:24.087752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403567+0000","last_change":"2024-09-16T10:33:19.869599+0000","last_active":"2024-09-16T10:33:24.403567+0000","last_peered":"2024-09-16T10:33:24.403567+0000","last_clean":"2024-09-16T10:33:24.403567+0000","last_became_active":"2024-09-16T10:33:19.869045+0000","last_became_peered":"2024-09-16T10:33:19.869045+0000","last_unstale":"2024-09-16T10:33:24.403567+0000","last_undegraded":"2024-09-16T10:33:24.403567+0000","last_fullsized":"2024-09-16T10:33:24.403567+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:50:10.557168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405282+0000","last_change":"2024-09-16T10:33:13.830662+0000","last_active":"2024-09-16T10:33:24.405282+0000","last_peered":"2024-09-16T10:33:24.405282+0000","last_clean":"2024-09-16T10:33:24.405282+0000","last_became_active":"2024-09-16T10:33:13.830216+0000","last_became_peered":"2024-09-16T10:33:13.830216+0000","last_unstale":"2024-09-16T10:33:24.405282+0000","last_undegraded":"2024-09-16T10:33:24.405282+0000","last_fullsized":"2024-09-16T10:33:24.405282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:01:37.779851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":27,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403555+0000","last_change":"2024-09-16T10:33:15.848652+0000","last_active":"2024-09-16T10:33:24.403555+0000","last_peered":"2024-09-16T10:33:24.403555+0000","last_clean":"2024-09-16T10:33:24.403555+0000","last_became_active":"2024-09-16T10:33:15.848338+0000","last_became_peered":"2024-09-16T10:33:15.848338+0000","last_unstale":"2024-09-16T10:33:24.403555+0000","last_undegraded":"2024-09-16T10:33:24.403555+0000","last_fullsized":"2024-09-16T10:33:24.403555+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:51.768572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403718+0000","last_change":"2024-09-16T10:33:17.859749+0000","last_active":"2024-09-16T10:33:24.403718+0000","last_peered":"2024-09-16T10:33:24.403718+0000","last_clean":"2024-09-16T10:33:24.403718+0000","last_became_active":"2024-09-16T10:33:17.859335+0000","last_became_peered":"2024-09-16T10:33:17.859335+0000","last_unstale":"2024-09-16T10:33:24.403718+0000","last_undegraded":"2024-09-16T10:33:24.403718+0000","last_fullsized":"2024-09-16T10:33:24.403718+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:47.684045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117298+0000","last_change":"2024-09-16T10:33:19.873926+0000","last_active":"2024-09-16T10:33:24.117298+0000","last_peered":"2024-09-16T10:33:24.117298+0000","last_clean":"2024-09-16T10:33:24.117298+0000","last_became_active":"2024-09-16T10:33:19.873588+0000","last_became_peered":"2024-09-16T10:33:19.873588+0000","last_unstale":"2024-09-16T10:33:24.117298+0000","last_undegraded":"2024-09-16T10:33:24.117298+0000","last_fullsized":"2024-09-16T10:33:24.117298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:31.453260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405282+0000","last_change":"2024-09-16T10:33:13.825016+0000","last_active":"2024-09-16T10:33:24.405282+0000","last_peered":"2024-09-16T10:33:24.405282+0000","last_clean":"2024-09-16T10:33:24.405282+0000","last_became_active":"2024-09-16T10:33:13.824739+0000","last_became_peered":"2024-09-16T10:33:13.824739+0000","last_unstale":"2024-09-16T10:33:24.405282+0000","last_undegraded":"2024-09-16T10:33:24.405282+0000","last_fullsized":"2024-09-16T10:33:24.405282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:46.420520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403760+0000","last_change":"2024-09-16T10:33:15.853868+0000","last_active":"2024-09-16T10:33:24.403760+0000","last_peered":"2024-09-16T10:33:24.403760+0000","last_clean":"2024-09-16T10:33:24.403760+0000","last_became_active":"2024-09-16T10:33:15.853552+0000","last_became_peered":"2024-09-16T10:33:15.853552+0000","last_unstale":"2024-09-16T10:33:24.403760+0000","last_undegraded":"2024-09-16T10:33:24.403760+0000","last_fullsized":"2024-09-16T10:33:24.403760+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:31.151079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117384+0000","last_change":"2024-09-16T10:33:17.857859+0000","last_active":"2024-09-16T10:33:24.117384+0000","last_peered":"2024-09-16T10:33:24.117384+0000","last_clean":"2024-09-16T10:33:24.117384+0000","last_became_active":"2024-09-16T10:33:17.857526+0000","last_became_peered":"2024-09-16T10:33:17.857526+0000","last_unstale":"2024-09-16T10:33:24.117384+0000","last_undegraded":"2024-09-16T10:33:24.117384+0000","last_fullsized":"2024-09-16T10:33:24.117384+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:07.015810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403877+0000","last_change":"2024-09-16T10:33:19.876901+0000","last_active":"2024-09-16T10:33:24.403877+0000","last_peered":"2024-09-16T10:33:24.403877+0000","last_clean":"2024-09-16T10:33:24.403877+0000","last_became_active":"2024-09-16T10:33:19.875179+0000","last_became_peered":"2024-09-16T10:33:19.875179+0000","last_unstale":"2024-09-16T10:33:24.403877+0000","last_undegraded":"2024-09-16T10:33:24.403877+0000","last_fullsized":"2024-09-16T10:33:24.403877+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:12.257683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":45,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405219+0000","last_change":"2024-09-16T10:33:15.845978+0000","last_active":"2024-09-16T10:33:24.405219+0000","last_peered":"2024-09-16T10:33:24.405219+0000","last_clean":"2024-09-16T10:33:24.405219+0000","last_became_active":"2024-09-16T10:33:15.845619+0000","last_became_peered":"2024-09-16T10:33:15.845619+0000","last_unstale":"2024-09-16T10:33:24.405219+0000","last_undegraded":"2024-09-16T10:33:24.405219+0000","last_fullsized":"2024-09-16T10:33:24.405219+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:24:07.328422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404998+0000","last_change":"2024-09-16T10:33:13.825242+0000","last_active":"2024-09-16T10:33:24.404998+0000","last_peered":"2024-09-16T10:33:24.404998+0000","last_clean":"2024-09-16T10:33:24.404998+0000","last_became_active":"2024-09-16T10:33:13.824994+0000","last_became_peered":"2024-09-16T10:33:13.824994+0000","last_unstale":"2024-09-16T10:33:24.404998+0000","last_undegraded":"2024-09-16T10:33:24.404998+0000","last_fullsized":"2024-09-16T10:33:24.404998+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:33.720530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121078+0000","last_change":"2024-09-16T10:33:17.857808+0000","last_active":"2024-09-16T10:33:24.121078+0000","last_peered":"2024-09-16T10:33:24.121078+0000","last_clean":"2024-09-16T10:33:24.121078+0000","last_became_active":"2024-09-16T10:33:17.857513+0000","last_became_peered":"2024-09-16T10:33:17.857513+0000","last_unstale":"2024-09-16T10:33:24.121078+0000","last_undegraded":"2024-09-16T10:33:24.121078+0000","last_fullsized":"2024-09-16T10:33:24.121078+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:10.733400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405141+0000","last_change":"2024-09-16T10:33:19.870176+0000","last_active":"2024-09-16T10:33:24.405141+0000","last_peered":"2024-09-16T10:33:24.405141+0000","last_clean":"2024-09-16T10:33:24.405141+0000","last_became_active":"2024-09-16T10:33:19.869907+0000","last_became_peered":"2024-09-16T10:33:19.869907+0000","last_unstale":"2024-09-16T10:33:24.405141+0000","last_undegraded":"2024-09-16T10:33:24.405141+0000","last_fullsized":"2024-09-16T10:33:24.405141+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:31:33.611070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404853+0000","last_change":"2024-09-16T10:33:15.853956+0000","last_active":"2024-09-16T10:33:24.404853+0000","last_peered":"2024-09-16T10:33:24.404853+0000","last_clean":"2024-09-16T10:33:24.404853+0000","last_became_active":"2024-09-16T10:33:15.853626+0000","last_became_peered":"2024-09-16T10:33:15.853626+0000","last_unstale":"2024-09-16T10:33:24.404853+0000","last_undegraded":"2024-09-16T10:33:24.404853+0000","last_fullsized":"2024-09-16T10:33:24.404853+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:51:44.841430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121127+0000","last_change":"2024-09-16T10:33:13.833976+0000","last_active":"2024-09-16T10:33:24.121127+0000","last_peered":"2024-09-16T10:33:24.121127+0000","last_clean":"2024-09-16T10:33:24.121127+0000","last_became_active":"2024-09-16T10:33:13.833041+0000","last_became_peered":"2024-09-16T10:33:13.833041+0000","last_unstale":"2024-09-16T10:33:24.121127+0000","last_undegraded":"2024-09-16T10:33:24.121127+0000","last_fullsized":"2024-09-16T10:33:24.121127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:56.080692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200626+0000","last_change":"2024-09-16T10:33:17.857392+0000","last_active":"2024-09-16T10:34:27.200626+0000","last_peered":"2024-09-16T10:34:27.200626+0000","last_clean":"2024-09-16T10:34:27.200626+0000","last_became_active":"2024-09-16T10:33:17.857052+0000","last_became_peered":"2024-09-16T10:33:17.857052+0000","last_unstale":"2024-09-16T10:34:27.200626+0000","last_undegraded":"2024-09-16T10:34:27.200626+0000","last_fullsized":"2024-09-16T10:34:27.200626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:44:19.221848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405032+0000","last_change":"2024-09-16T10:33:19.866574+0000","last_active":"2024-09-16T10:33:24.405032+0000","last_peered":"2024-09-16T10:33:24.405032+0000","last_clean":"2024-09-16T10:33:24.405032+0000","last_became_active":"2024-09-16T10:33:19.866300+0000","last_became_peered":"2024-09-16T10:33:19.866300+0000","last_unstale":"2024-09-16T10:33:24.405032+0000","last_undegraded":"2024-09-16T10:33:24.405032+0000","last_fullsized":"2024-09-16T10:33:24.405032+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:28:46.742763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122075+0000","last_change":"2024-09-16T10:33:15.852171+0000","last_active":"2024-09-16T10:33:24.122075+0000","last_peered":"2024-09-16T10:33:24.122075+0000","last_clean":"2024-09-16T10:33:24.122075+0000","last_became_active":"2024-09-16T10:33:15.850300+0000","last_became_peered":"2024-09-16T10:33:15.850300+0000","last_unstale":"2024-09-16T10:33:24.122075+0000","last_undegraded":"2024-09-16T10:33:24.122075+0000","last_fullsized":"2024-09-16T10:33:24.122075+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:32.967386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120794+0000","last_change":"2024-09-16T10:33:13.834432+0000","last_active":"2024-09-16T10:33:24.120794+0000","last_peered":"2024-09-16T10:33:24.120794+0000","last_clean":"2024-09-16T10:33:24.120794+0000","last_became_active":"2024-09-16T10:33:13.833975+0000","last_became_peered":"2024-09-16T10:33:13.833975+0000","last_unstale":"2024-09-16T10:33:24.120794+0000","last_undegraded":"2024-09-16T10:33:24.120794+0000","last_fullsized":"2024-09-16T10:33:24.120794+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:55.639735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200708+0000","last_change":"2024-09-16T10:33:17.858688+0000","last_active":"2024-09-16T10:34:27.200708+0000","last_peered":"2024-09-16T10:34:27.200708+0000","last_clean":"2024-09-16T10:34:27.200708+0000","last_became_active":"2024-09-16T10:33:17.858326+0000","last_became_peered":"2024-09-16T10:33:17.858326+0000","last_unstale":"2024-09-16T10:34:27.200708+0000","last_undegraded":"2024-09-16T10:34:27.200708+0000","last_fullsized":"2024-09-16T10:34:27.200708+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:15.333230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117618+0000","last_change":"2024-09-16T10:33:19.871992+0000","last_active":"2024-09-16T10:33:24.117618+0000","last_peered":"2024-09-16T10:33:24.117618+0000","last_clean":"2024-09-16T10:33:24.117618+0000","last_became_active":"2024-09-16T10:33:19.871755+0000","last_became_peered":"2024-09-16T10:33:19.871755+0000","last_unstale":"2024-09-16T10:33:24.117618+0000","last_undegraded":"2024-09-16T10:33:24.117618+0000","last_fullsized":"2024-09-16T10:33:24.117618+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:18.251978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405269+0000","last_change":"2024-09-16T10:33:15.848251+0000","last_active":"2024-09-16T10:33:24.405269+0000","last_peered":"2024-09-16T10:33:24.405269+0000","last_clean":"2024-09-16T10:33:24.405269+0000","last_became_active":"2024-09-16T10:33:15.847981+0000","last_became_peered":"2024-09-16T10:33:15.847981+0000","last_unstale":"2024-09-16T10:33:24.405269+0000","last_undegraded":"2024-09-16T10:33:24.405269+0000","last_fullsized":"2024-09-16T10:33:24.405269+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:44.218431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403636+0000","last_change":"2024-09-16T10:33:13.830092+0000","last_active":"2024-09-16T10:33:24.403636+0000","last_peered":"2024-09-16T10:33:24.403636+0000","last_clean":"2024-09-16T10:33:24.403636+0000","last_became_active":"2024-09-16T10:33:13.829790+0000","last_became_peered":"2024-09-16T10:33:13.829790+0000","last_unstale":"2024-09-16T10:33:24.403636+0000","last_undegraded":"2024-09-16T10:33:24.403636+0000","last_fullsized":"2024-09-16T10:33:24.403636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:39.118186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117804+0000","last_change":"2024-09-16T10:33:17.862965+0000","last_active":"2024-09-16T10:33:24.117804+0000","last_peered":"2024-09-16T10:33:24.117804+0000","last_clean":"2024-09-16T10:33:24.117804+0000","last_became_active":"2024-09-16T10:33:17.862684+0000","last_became_peered":"2024-09-16T10:33:17.862684+0000","last_unstale":"2024-09-16T10:33:24.117804+0000","last_undegraded":"2024-09-16T10:33:24.117804+0000","last_fullsized":"2024-09-16T10:33:24.117804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:27.999100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122000+0000","last_change":"2024-09-16T10:33:19.908083+0000","last_active":"2024-09-16T10:33:24.122000+0000","last_peered":"2024-09-16T10:33:24.122000+0000","last_clean":"2024-09-16T10:33:24.122000+0000","last_became_active":"2024-09-16T10:33:19.907721+0000","last_became_peered":"2024-09-16T10:33:19.907721+0000","last_unstale":"2024-09-16T10:33:24.122000+0000","last_undegraded":"2024-09-16T10:33:24.122000+0000","last_fullsized":"2024-09-16T10:33:24.122000+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:06:18.374644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":38,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404409+0000","last_change":"2024-09-16T10:33:15.847080+0000","last_active":"2024-09-16T10:33:24.404409+0000","last_peered":"2024-09-16T10:33:24.404409+0000","last_clean":"2024-09-16T10:33:24.404409+0000","last_became_active":"2024-09-16T10:33:15.846198+0000","last_became_peered":"2024-09-16T10:33:15.846198+0000","last_unstale":"2024-09-16T10:33:24.404409+0000","last_undegraded":"2024-09-16T10:33:24.404409+0000","last_fullsized":"2024-09-16T10:33:24.404409+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:31.743731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404227+0000","last_change":"2024-09-16T10:33:13.823815+0000","last_active":"2024-09-16T10:33:24.404227+0000","last_peered":"2024-09-16T10:33:24.404227+0000","last_clean":"2024-09-16T10:33:24.404227+0000","last_became_active":"2024-09-16T10:33:13.823321+0000","last_became_peered":"2024-09-16T10:33:13.823321+0000","last_unstale":"2024-09-16T10:33:24.404227+0000","last_undegraded":"2024-09-16T10:33:24.404227+0000","last_fullsized":"2024-09-16T10:33:24.404227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:22.783191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200611+0000","last_change":"2024-09-16T10:33:17.858121+0000","last_active":"2024-09-16T10:34:27.200611+0000","last_peered":"2024-09-16T10:34:27.200611+0000","last_clean":"2024-09-16T10:34:27.200611+0000","last_became_active":"2024-09-16T10:33:17.857853+0000","last_became_peered":"2024-09-16T10:33:17.857853+0000","last_unstale":"2024-09-16T10:34:27.200611+0000","last_undegraded":"2024-09-16T10:34:27.200611+0000","last_fullsized":"2024-09-16T10:34:27.200611+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:59.348644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404658+0000","last_change":"2024-09-16T10:33:19.909477+0000","last_active":"2024-09-16T10:33:24.404658+0000","last_peered":"2024-09-16T10:33:24.404658+0000","last_clean":"2024-09-16T10:33:24.404658+0000","last_became_active":"2024-09-16T10:33:19.909194+0000","last_became_peered":"2024-09-16T10:33:19.909194+0000","last_unstale":"2024-09-16T10:33:24.404658+0000","last_undegraded":"2024-09-16T10:33:24.404658+0000","last_fullsized":"2024-09-16T10:33:24.404658+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:13.158503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":49,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121523+0000","last_change":"2024-09-16T10:33:15.850983+0000","last_active":"2024-09-16T10:33:24.121523+0000","last_peered":"2024-09-16T10:33:24.121523+0000","last_clean":"2024-09-16T10:33:24.121523+0000","last_became_active":"2024-09-16T10:33:15.850720+0000","last_became_peered":"2024-09-16T10:33:15.850720+0000","last_unstale":"2024-09-16T10:33:24.121523+0000","last_undegraded":"2024-09-16T10:33:24.121523+0000","last_fullsized":"2024-09-16T10:33:24.121523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:57.092520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122722+0000","last_change":"2024-09-16T10:33:13.821362+0000","last_active":"2024-09-16T10:33:24.122722+0000","last_peered":"2024-09-16T10:33:24.122722+0000","last_clean":"2024-09-16T10:33:24.122722+0000","last_became_active":"2024-09-16T10:33:13.821102+0000","last_became_peered":"2024-09-16T10:33:13.821102+0000","last_unstale":"2024-09-16T10:33:24.122722+0000","last_undegraded":"2024-09-16T10:33:24.122722+0000","last_fullsized":"2024-09-16T10:33:24.122722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:41:38.478745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404320+0000","last_change":"2024-09-16T10:33:17.865528+0000","last_active":"2024-09-16T10:33:24.404320+0000","last_peered":"2024-09-16T10:33:24.404320+0000","last_clean":"2024-09-16T10:33:24.404320+0000","last_became_active":"2024-09-16T10:33:17.865249+0000","last_became_peered":"2024-09-16T10:33:17.865249+0000","last_unstale":"2024-09-16T10:33:24.404320+0000","last_undegraded":"2024-09-16T10:33:24.404320+0000","last_fullsized":"2024-09-16T10:33:24.404320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:08:27.575571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403753+0000","last_change":"2024-09-16T10:33:19.872124+0000","last_active":"2024-09-16T10:33:24.403753+0000","last_peered":"2024-09-16T10:33:24.403753+0000","last_clean":"2024-09-16T10:33:24.403753+0000","last_became_active":"2024-09-16T10:33:19.871812+0000","last_became_peered":"2024-09-16T10:33:19.871812+0000","last_unstale":"2024-09-16T10:33:24.403753+0000","last_undegraded":"2024-09-16T10:33:24.403753+0000","last_fullsized":"2024-09-16T10:33:24.403753+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:54:05.585783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":53,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404700+0000","last_change":"2024-09-16T10:33:15.846613+0000","last_active":"2024-09-16T10:33:24.404700+0000","last_peered":"2024-09-16T10:33:24.404700+0000","last_clean":"2024-09-16T10:33:24.404700+0000","last_became_active":"2024-09-16T10:33:15.846321+0000","last_became_peered":"2024-09-16T10:33:15.846321+0000","last_unstale":"2024-09-16T10:33:24.404700+0000","last_undegraded":"2024-09-16T10:33:24.404700+0000","last_fullsized":"2024-09-16T10:33:24.404700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:38.391756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405358+0000","last_change":"2024-09-16T10:33:13.830471+0000","last_active":"2024-09-16T10:33:24.405358+0000","last_peered":"2024-09-16T10:33:24.405358+0000","last_clean":"2024-09-16T10:33:24.405358+0000","last_became_active":"2024-09-16T10:33:13.829598+0000","last_became_peered":"2024-09-16T10:33:13.829598+0000","last_unstale":"2024-09-16T10:33:24.405358+0000","last_undegraded":"2024-09-16T10:33:24.405358+0000","last_fullsized":"2024-09-16T10:33:24.405358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:55.836948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405167+0000","last_change":"2024-09-16T10:33:17.858857+0000","last_active":"2024-09-16T10:33:24.405167+0000","last_peered":"2024-09-16T10:33:24.405167+0000","last_clean":"2024-09-16T10:33:24.405167+0000","last_became_active":"2024-09-16T10:33:17.858598+0000","last_became_peered":"2024-09-16T10:33:17.858598+0000","last_unstale":"2024-09-16T10:33:24.405167+0000","last_undegraded":"2024-09-16T10:33:24.405167+0000","last_fullsized":"2024-09-16T10:33:24.405167+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:48.093085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120387+0000","last_change":"2024-09-16T10:33:19.907978+0000","last_active":"2024-09-16T10:33:24.120387+0000","last_peered":"2024-09-16T10:33:24.120387+0000","last_clean":"2024-09-16T10:33:24.120387+0000","last_became_active":"2024-09-16T10:33:19.907676+0000","last_became_peered":"2024-09-16T10:33:19.907676+0000","last_unstale":"2024-09-16T10:33:24.120387+0000","last_undegraded":"2024-09-16T10:33:24.120387+0000","last_fullsized":"2024-09-16T10:33:24.120387+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:45.261026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120992+0000","last_change":"2024-09-16T10:33:15.845291+0000","last_active":"2024-09-16T10:33:24.120992+0000","last_peered":"2024-09-16T10:33:24.120992+0000","last_clean":"2024-09-16T10:33:24.120992+0000","last_became_active":"2024-09-16T10:33:15.845011+0000","last_became_peered":"2024-09-16T10:33:15.845011+0000","last_unstale":"2024-09-16T10:33:24.120992+0000","last_undegraded":"2024-09-16T10:33:24.120992+0000","last_fullsized":"2024-09-16T10:33:24.120992+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:31.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121261+0000","last_change":"2024-09-16T10:33:13.827724+0000","last_active":"2024-09-16T10:33:24.121261+0000","last_peered":"2024-09-16T10:33:24.121261+0000","last_clean":"2024-09-16T10:33:24.121261+0000","last_became_active":"2024-09-16T10:33:13.827401+0000","last_became_peered":"2024-09-16T10:33:13.827401+0000","last_unstale":"2024-09-16T10:33:24.121261+0000","last_undegraded":"2024-09-16T10:33:24.121261+0000","last_fullsized":"2024-09-16T10:33:24.121261+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:59.906669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200591+0000","last_change":"2024-09-16T10:33:17.859954+0000","last_active":"2024-09-16T10:34:27.200591+0000","last_peered":"2024-09-16T10:34:27.200591+0000","last_clean":"2024-09-16T10:34:27.200591+0000","last_became_active":"2024-09-16T10:33:17.859634+0000","last_became_peered":"2024-09-16T10:33:17.859634+0000","last_unstale":"2024-09-16T10:34:27.200591+0000","last_undegraded":"2024-09-16T10:34:27.200591+0000","last_fullsized":"2024-09-16T10:34:27.200591+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:26.862187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403725+0000","last_change":"2024-09-16T10:33:19.877161+0000","last_active":"2024-09-16T10:33:24.403725+0000","last_peered":"2024-09-16T10:33:24.403725+0000","last_clean":"2024-09-16T10:33:24.403725+0000","last_became_active":"2024-09-16T10:33:19.876609+0000","last_became_peered":"2024-09-16T10:33:19.876609+0000","last_unstale":"2024-09-16T10:33:24.403725+0000","last_undegraded":"2024-09-16T10:33:24.403725+0000","last_fullsized":"2024-09-16T10:33:24.403725+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:08:43.855631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":81,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200607+0000","last_change":"2024-09-16T10:33:15.850954+0000","last_active":"2024-09-16T10:34:27.200607+0000","last_peered":"2024-09-16T10:34:27.200607+0000","last_clean":"2024-09-16T10:34:27.200607+0000","last_became_active":"2024-09-16T10:33:15.850147+0000","last_became_peered":"2024-09-16T10:33:15.850147+0000","last_unstale":"2024-09-16T10:34:27.200607+0000","last_undegraded":"2024-09-16T10:34:27.200607+0000","last_fullsized":"2024-09-16T10:34:27.200607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:41.061633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117540+0000","last_change":"2024-09-16T10:33:13.826857+0000","last_active":"2024-09-16T10:33:24.117540+0000","last_peered":"2024-09-16T10:33:24.117540+0000","last_clean":"2024-09-16T10:33:24.117540+0000","last_became_active":"2024-09-16T10:33:13.826596+0000","last_became_peered":"2024-09-16T10:33:13.826596+0000","last_unstale":"2024-09-16T10:33:24.117540+0000","last_undegraded":"2024-09-16T10:33:24.117540+0000","last_fullsized":"2024-09-16T10:33:24.117540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:52.557415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404482+0000","last_change":"2024-09-16T10:33:17.863382+0000","last_active":"2024-09-16T10:33:24.404482+0000","last_peered":"2024-09-16T10:33:24.404482+0000","last_clean":"2024-09-16T10:33:24.404482+0000","last_became_active":"2024-09-16T10:33:17.863013+0000","last_became_peered":"2024-09-16T10:33:17.863013+0000","last_unstale":"2024-09-16T10:33:24.404482+0000","last_undegraded":"2024-09-16T10:33:24.404482+0000","last_fullsized":"2024-09-16T10:33:24.404482+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:29:07.651347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122091+0000","last_change":"2024-09-16T10:33:19.872874+0000","last_active":"2024-09-16T10:33:24.122091+0000","last_peered":"2024-09-16T10:33:24.122091+0000","last_clean":"2024-09-16T10:33:24.122091+0000","last_became_active":"2024-09-16T10:33:19.872557+0000","last_became_peered":"2024-09-16T10:33:19.872557+0000","last_unstale":"2024-09-16T10:33:24.122091+0000","last_undegraded":"2024-09-16T10:33:24.122091+0000","last_fullsized":"2024-09-16T10:33:24.122091+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:16.485585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":44,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122158+0000","last_change":"2024-09-16T10:33:15.849555+0000","last_active":"2024-09-16T10:33:24.122158+0000","last_peered":"2024-09-16T10:33:24.122158+0000","last_clean":"2024-09-16T10:33:24.122158+0000","last_became_active":"2024-09-16T10:33:15.848762+0000","last_became_peered":"2024-09-16T10:33:15.848762+0000","last_unstale":"2024-09-16T10:33:24.122158+0000","last_undegraded":"2024-09-16T10:33:24.122158+0000","last_fullsized":"2024-09-16T10:33:24.122158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:47:06.625488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"69'5","reported_seq":101,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:33.388605+0000","last_change":"2024-09-16T10:33:17.848105+0000","last_active":"2024-09-16T10:34:33.388605+0000","last_peered":"2024-09-16T10:34:33.388605+0000","last_clean":"2024-09-16T10:34:33.388605+0000","last_became_active":"2024-09-16T10:33:13.819415+0000","last_became_peered":"2024-09-16T10:33:13.819415+0000","last_unstale":"2024-09-16T10:34:33.388605+0000","last_undegraded":"2024-09-16T10:34:33.388605+0000","last_fullsized":"2024-09-16T10:34:33.388605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:54.771452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002177357,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404364+0000","last_change":"2024-09-16T10:33:13.830338+0000","last_active":"2024-09-16T10:33:24.404364+0000","last_peered":"2024-09-16T10:33:24.404364+0000","last_clean":"2024-09-16T10:33:24.404364+0000","last_became_active":"2024-09-16T10:33:13.830063+0000","last_became_peered":"2024-09-16T10:33:13.830063+0000","last_unstale":"2024-09-16T10:33:24.404364+0000","last_undegraded":"2024-09-16T10:33:24.404364+0000","last_fullsized":"2024-09-16T10:33:24.404364+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:17:38.454962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404921+0000","last_change":"2024-09-16T10:33:17.856564+0000","last_active":"2024-09-16T10:33:24.404921+0000","last_peered":"2024-09-16T10:33:24.404921+0000","last_clean":"2024-09-16T10:33:24.404921+0000","last_became_active":"2024-09-16T10:33:17.856131+0000","last_became_peered":"2024-09-16T10:33:17.856131+0000","last_unstale":"2024-09-16T10:33:24.404921+0000","last_undegraded":"2024-09-16T10:33:24.404921+0000","last_fullsized":"2024-09-16T10:33:24.404921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:19.224434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405056+0000","last_change":"2024-09-16T10:33:19.907992+0000","last_active":"2024-09-16T10:33:24.405056+0000","last_peered":"2024-09-16T10:33:24.405056+0000","last_clean":"2024-09-16T10:33:24.405056+0000","last_became_active":"2024-09-16T10:33:19.907706+0000","last_became_peered":"2024-09-16T10:33:19.907706+0000","last_unstale":"2024-09-16T10:33:24.405056+0000","last_undegraded":"2024-09-16T10:33:24.405056+0000","last_fullsized":"2024-09-16T10:33:24.405056+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:51:51.114352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":35,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405386+0000","last_change":"2024-09-16T10:33:15.846739+0000","last_active":"2024-09-16T10:33:24.405386+0000","last_peered":"2024-09-16T10:33:24.405386+0000","last_clean":"2024-09-16T10:33:24.405386+0000","last_became_active":"2024-09-16T10:33:15.846017+0000","last_became_peered":"2024-09-16T10:33:15.846017+0000","last_unstale":"2024-09-16T10:33:24.405386+0000","last_undegraded":"2024-09-16T10:33:24.405386+0000","last_fullsized":"2024-09-16T10:33:24.405386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:33.275546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'1","reported_seq":25,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404363+0000","last_change":"2024-09-16T10:33:17.852278+0000","last_active":"2024-09-16T10:33:24.404363+0000","last_peered":"2024-09-16T10:33:24.404363+0000","last_clean":"2024-09-16T10:33:24.404363+0000","last_became_active":"2024-09-16T10:33:13.833909+0000","last_became_peered":"2024-09-16T10:33:13.833909+0000","last_unstale":"2024-09-16T10:33:24.404363+0000","last_undegraded":"2024-09-16T10:33:24.404363+0000","last_fullsized":"2024-09-16T10:33:24.404363+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:08.748999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037669100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'1","reported_seq":24,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403712+0000","last_change":"2024-09-16T10:33:13.829887+0000","last_active":"2024-09-16T10:33:24.403712+0000","last_peered":"2024-09-16T10:33:24.403712+0000","last_clean":"2024-09-16T10:33:24.403712+0000","last_became_active":"2024-09-16T10:33:13.829591+0000","last_became_peered":"2024-09-16T10:33:13.829591+0000","last_unstale":"2024-09-16T10:33:24.403712+0000","last_undegraded":"2024-09-16T10:33:24.403712+0000","last_fullsized":"2024-09-16T10:33:24.403712+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:28.484307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117828+0000","last_change":"2024-09-16T10:33:17.860155+0000","last_active":"2024-09-16T10:33:24.117828+0000","last_peered":"2024-09-16T10:33:24.117828+0000","last_clean":"2024-09-16T10:33:24.117828+0000","last_became_active":"2024-09-16T10:33:17.859769+0000","last_became_peered":"2024-09-16T10:33:17.859769+0000","last_unstale":"2024-09-16T10:33:24.117828+0000","last_undegraded":"2024-09-16T10:33:24.117828+0000","last_fullsized":"2024-09-16T10:33:24.117828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:30.480504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"68'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120990+0000","last_change":"2024-09-16T10:33:19.873795+0000","last_active":"2024-09-16T10:33:24.120990+0000","last_peered":"2024-09-16T10:33:24.120990+0000","last_clean":"2024-09-16T10:33:24.120990+0000","last_became_active":"2024-09-16T10:33:19.873433+0000","last_became_peered":"2024-09-16T10:33:19.873433+0000","last_unstale":"2024-09-16T10:33:24.120990+0000","last_undegraded":"2024-09-16T10:33:24.120990+0000","last_fullsized":"2024-09-16T10:33:24.120990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:20:54.213371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"68'14","reported_seq":39,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404285+0000","last_change":"2024-09-16T10:33:15.844147+0000","last_active":"2024-09-16T10:33:24.404285+0000","last_peered":"2024-09-16T10:33:24.404285+0000","last_clean":"2024-09-16T10:33:24.404285+0000","last_became_active":"2024-09-16T10:33:15.843754+0000","last_became_peered":"2024-09-16T10:33:15.843754+0000","last_unstale":"2024-09-16T10:33:24.404285+0000","last_undegraded":"2024-09-16T10:33:24.404285+0000","last_fullsized":"2024-09-16T10:33:24.404285+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:17.719923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122103+0000","last_change":"2024-09-16T10:33:17.857138+0000","last_active":"2024-09-16T10:33:24.122103+0000","last_peered":"2024-09-16T10:33:24.122103+0000","last_clean":"2024-09-16T10:33:24.122103+0000","last_became_active":"2024-09-16T10:33:13.829882+0000","last_became_peered":"2024-09-16T10:33:13.829882+0000","last_unstale":"2024-09-16T10:33:24.122103+0000","last_undegraded":"2024-09-16T10:33:24.122103+0000","last_fullsized":"2024-09-16T10:33:24.122103+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:48:49.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018861469999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117501+0000","last_change":"2024-09-16T10:33:13.832221+0000","last_active":"2024-09-16T10:33:24.117501+0000","last_peered":"2024-09-16T10:33:24.117501+0000","last_clean":"2024-09-16T10:33:24.117501+0000","last_became_active":"2024-09-16T10:33:13.831955+0000","last_became_peered":"2024-09-16T10:33:13.831955+0000","last_unstale":"2024-09-16T10:33:24.117501+0000","last_undegraded":"2024-09-16T10:33:24.117501+0000","last_fullsized":"2024-09-16T10:33:24.117501+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:07.925458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'188","reported_seq":334,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:33:51.058979+0000","last_change":"2024-09-16T10:33:02.009627+0000","last_active":"2024-09-16T10:33:51.058979+0000","last_peered":"2024-09-16T10:33:51.058979+0000","last_clean":"2024-09-16T10:33:51.058979+0000","last_became_active":"2024-09-16T10:33:02.009112+0000","last_became_peered":"2024-09-16T10:33:02.009112+0000","last_unstale":"2024-09-16T10:33:51.058979+0000","last_undegraded":"2024-09-16T10:33:51.058979+0000","last_fullsized":"2024-09-16T10:33:51.058979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:31:00.046657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:31:00.046657+0000","last_clean_scrub_stamp":"2024-09-16T10:31:00.046657+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:51.264238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405068+0000","last_change":"2024-09-16T10:33:17.856219+0000","last_active":"2024-09-16T10:33:24.405068+0000","last_peered":"2024-09-16T10:33:24.405068+0000","last_clean":"2024-09-16T10:33:24.405068+0000","last_became_active":"2024-09-16T10:33:17.855883+0000","last_became_peered":"2024-09-16T10:33:17.855883+0000","last_unstale":"2024-09-16T10:33:24.405068+0000","last_undegraded":"2024-09-16T10:33:24.405068+0000","last_fullsized":"2024-09-16T10:33:24.405068+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:04:09.771912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117876+0000","last_change":"2024-09-16T10:33:19.870198+0000","last_active":"2024-09-16T10:33:24.117876+0000","last_peered":"2024-09-16T10:33:24.117876+0000","last_clean":"2024-09-16T10:33:24.117876+0000","last_became_active":"2024-09-16T10:33:19.869936+0000","last_became_peered":"2024-09-16T10:33:19.869936+0000","last_unstale":"2024-09-16T10:33:24.117876+0000","last_undegraded":"2024-09-16T10:33:24.117876+0000","last_fullsized":"2024-09-16T10:33:24.117876+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:26.776086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":48,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405339+0000","last_change":"2024-09-16T10:33:15.845378+0000","last_active":"2024-09-16T10:33:24.405339+0000","last_peered":"2024-09-16T10:33:24.405339+0000","last_clean":"2024-09-16T10:33:24.405339+0000","last_became_active":"2024-09-16T10:33:15.844800+0000","last_became_peered":"2024-09-16T10:33:15.844800+0000","last_unstale":"2024-09-16T10:33:24.405339+0000","last_undegraded":"2024-09-16T10:33:24.405339+0000","last_fullsized":"2024-09-16T10:33:24.405339+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:05.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404921+0000","last_change":"2024-09-16T10:33:13.827575+0000","last_active":"2024-09-16T10:33:24.404921+0000","last_peered":"2024-09-16T10:33:24.404921+0000","last_clean":"2024-09-16T10:33:24.404921+0000","last_became_active":"2024-09-16T10:33:13.827277+0000","last_became_peered":"2024-09-16T10:33:13.827277+0000","last_unstale":"2024-09-16T10:33:24.404921+0000","last_undegraded":"2024-09-16T10:33:24.404921+0000","last_fullsized":"2024-09-16T10:33:24.404921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:34:59.152384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404298+0000","last_change":"2024-09-16T10:33:17.858189+0000","last_active":"2024-09-16T10:33:24.404298+0000","last_peered":"2024-09-16T10:33:24.404298+0000","last_clean":"2024-09-16T10:33:24.404298+0000","last_became_active":"2024-09-16T10:33:17.857563+0000","last_became_peered":"2024-09-16T10:33:17.857563+0000","last_unstale":"2024-09-16T10:33:24.404298+0000","last_undegraded":"2024-09-16T10:33:24.404298+0000","last_fullsized":"2024-09-16T10:33:24.404298+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:31.954377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403799+0000","last_change":"2024-09-16T10:33:19.872038+0000","last_active":"2024-09-16T10:33:24.403799+0000","last_peered":"2024-09-16T10:33:24.403799+0000","last_clean":"2024-09-16T10:33:24.403799+0000","last_became_active":"2024-09-16T10:33:19.871772+0000","last_became_peered":"2024-09-16T10:33:19.871772+0000","last_unstale":"2024-09-16T10:33:24.403799+0000","last_undegraded":"2024-09-16T10:33:24.403799+0000","last_fullsized":"2024-09-16T10:33:24.403799+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:39.278027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404591+0000","last_change":"2024-09-16T10:33:15.848259+0000","last_active":"2024-09-16T10:33:24.404591+0000","last_peered":"2024-09-16T10:33:24.404591+0000","last_clean":"2024-09-16T10:33:24.404591+0000","last_became_active":"2024-09-16T10:33:15.847953+0000","last_became_peered":"2024-09-16T10:33:15.847953+0000","last_unstale":"2024-09-16T10:33:24.404591+0000","last_undegraded":"2024-09-16T10:33:24.404591+0000","last_fullsized":"2024-09-16T10:33:24.404591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:39:39.168243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122005+0000","last_change":"2024-09-16T10:33:13.824507+0000","last_active":"2024-09-16T10:33:24.122005+0000","last_peered":"2024-09-16T10:33:24.122005+0000","last_clean":"2024-09-16T10:33:24.122005+0000","last_became_active":"2024-09-16T10:33:13.824098+0000","last_became_peered":"2024-09-16T10:33:13.824098+0000","last_unstale":"2024-09-16T10:33:24.122005+0000","last_undegraded":"2024-09-16T10:33:24.122005+0000","last_fullsized":"2024-09-16T10:33:24.122005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:53.987321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404118+0000","last_change":"2024-09-16T10:33:17.861312+0000","last_active":"2024-09-16T10:33:24.404118+0000","last_peered":"2024-09-16T10:33:24.404118+0000","last_clean":"2024-09-16T10:33:24.404118+0000","last_became_active":"2024-09-16T10:33:17.861006+0000","last_became_peered":"2024-09-16T10:33:17.861006+0000","last_unstale":"2024-09-16T10:33:24.404118+0000","last_undegraded":"2024-09-16T10:33:24.404118+0000","last_fullsized":"2024-09-16T10:33:24.404118+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:11:22.858849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404202+0000","last_change":"2024-09-16T10:33:19.876459+0000","last_active":"2024-09-16T10:33:24.404202+0000","last_peered":"2024-09-16T10:33:24.404202+0000","last_clean":"2024-09-16T10:33:24.404202+0000","last_became_active":"2024-09-16T10:33:19.876171+0000","last_became_peered":"2024-09-16T10:33:19.876171+0000","last_unstale":"2024-09-16T10:33:24.404202+0000","last_undegraded":"2024-09-16T10:33:24.404202+0000","last_fullsized":"2024-09-16T10:33:24.404202+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:47.206200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117195+0000","last_change":"2024-09-16T10:33:15.846052+0000","last_active":"2024-09-16T10:33:24.117195+0000","last_peered":"2024-09-16T10:33:24.117195+0000","last_clean":"2024-09-16T10:33:24.117195+0000","last_became_active":"2024-09-16T10:33:15.845756+0000","last_became_peered":"2024-09-16T10:33:15.845756+0000","last_unstale":"2024-09-16T10:33:24.117195+0000","last_undegraded":"2024-09-16T10:33:24.117195+0000","last_fullsized":"2024-09-16T10:33:24.117195+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:35.349667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"61'1","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405044+0000","last_change":"2024-09-16T10:33:13.823475+0000","last_active":"2024-09-16T10:33:24.405044+0000","last_peered":"2024-09-16T10:33:24.405044+0000","last_clean":"2024-09-16T10:33:24.405044+0000","last_became_active":"2024-09-16T10:33:13.823203+0000","last_became_peered":"2024-09-16T10:33:13.823203+0000","last_unstale":"2024-09-16T10:33:24.405044+0000","last_undegraded":"2024-09-16T10:33:24.405044+0000","last_fullsized":"2024-09-16T10:33:24.405044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:40.846895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200728+0000","last_change":"2024-09-16T10:33:17.859546+0000","last_active":"2024-09-16T10:34:27.200728+0000","last_peered":"2024-09-16T10:34:27.200728+0000","last_clean":"2024-09-16T10:34:27.200728+0000","last_became_active":"2024-09-16T10:33:17.859246+0000","last_became_peered":"2024-09-16T10:33:17.859246+0000","last_unstale":"2024-09-16T10:34:27.200728+0000","last_undegraded":"2024-09-16T10:34:27.200728+0000","last_fullsized":"2024-09-16T10:34:27.200728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:59.378380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117270+0000","last_change":"2024-09-16T10:33:19.908097+0000","last_active":"2024-09-16T10:33:24.117270+0000","last_peered":"2024-09-16T10:33:24.117270+0000","last_clean":"2024-09-16T10:33:24.117270+0000","last_became_active":"2024-09-16T10:33:19.907747+0000","last_became_peered":"2024-09-16T10:33:19.907747+0000","last_unstale":"2024-09-16T10:33:24.117270+0000","last_undegraded":"2024-09-16T10:33:24.117270+0000","last_fullsized":"2024-09-16T10:33:24.117270+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:29.122676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":40,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404318+0000","last_change":"2024-09-16T10:33:15.854009+0000","last_active":"2024-09-16T10:33:24.404318+0000","last_peered":"2024-09-16T10:33:24.404318+0000","last_clean":"2024-09-16T10:33:24.404318+0000","last_became_active":"2024-09-16T10:33:15.853645+0000","last_became_peered":"2024-09-16T10:33:15.853645+0000","last_unstale":"2024-09-16T10:33:24.404318+0000","last_undegraded":"2024-09-16T10:33:24.404318+0000","last_fullsized":"2024-09-16T10:33:24.404318+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:03.443239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404877+0000","last_change":"2024-09-16T10:33:13.828001+0000","last_active":"2024-09-16T10:33:24.404877+0000","last_peered":"2024-09-16T10:33:24.404877+0000","last_clean":"2024-09-16T10:33:24.404877+0000","last_became_active":"2024-09-16T10:33:13.827735+0000","last_became_peered":"2024-09-16T10:33:13.827735+0000","last_unstale":"2024-09-16T10:33:24.404877+0000","last_undegraded":"2024-09-16T10:33:24.404877+0000","last_fullsized":"2024-09-16T10:33:24.404877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:10.056999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403827+0000","last_change":"2024-09-16T10:33:17.854722+0000","last_active":"2024-09-16T10:33:24.403827+0000","last_peered":"2024-09-16T10:33:24.403827+0000","last_clean":"2024-09-16T10:33:24.403827+0000","last_became_active":"2024-09-16T10:33:17.854452+0000","last_became_peered":"2024-09-16T10:33:17.854452+0000","last_unstale":"2024-09-16T10:33:24.403827+0000","last_undegraded":"2024-09-16T10:33:24.403827+0000","last_fullsized":"2024-09-16T10:33:24.403827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:38:29.759276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122490+0000","last_change":"2024-09-16T10:33:19.875323+0000","last_active":"2024-09-16T10:33:24.122490+0000","last_peered":"2024-09-16T10:33:24.122490+0000","last_clean":"2024-09-16T10:33:24.122490+0000","last_became_active":"2024-09-16T10:33:19.874868+0000","last_became_peered":"2024-09-16T10:33:19.874868+0000","last_unstale":"2024-09-16T10:33:24.122490+0000","last_undegraded":"2024-09-16T10:33:24.122490+0000","last_fullsized":"2024-09-16T10:33:24.122490+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:07.941852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405359+0000","last_change":"2024-09-16T10:33:15.854049+0000","last_active":"2024-09-16T10:33:24.405359+0000","last_peered":"2024-09-16T10:33:24.405359+0000","last_clean":"2024-09-16T10:33:24.405359+0000","last_became_active":"2024-09-16T10:33:15.853789+0000","last_became_peered":"2024-09-16T10:33:15.853789+0000","last_unstale":"2024-09-16T10:33:24.405359+0000","last_undegraded":"2024-09-16T10:33:24.405359+0000","last_fullsized":"2024-09-16T10:33:24.405359+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:04.753452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403872+0000","last_change":"2024-09-16T10:33:13.827742+0000","last_active":"2024-09-16T10:33:24.403872+0000","last_peered":"2024-09-16T10:33:24.403872+0000","last_clean":"2024-09-16T10:33:24.403872+0000","last_became_active":"2024-09-16T10:33:13.827292+0000","last_became_peered":"2024-09-16T10:33:13.827292+0000","last_unstale":"2024-09-16T10:33:24.403872+0000","last_undegraded":"2024-09-16T10:33:24.403872+0000","last_fullsized":"2024-09-16T10:33:24.403872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:23.812314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121732+0000","last_change":"2024-09-16T10:33:17.857595+0000","last_active":"2024-09-16T10:33:24.121732+0000","last_peered":"2024-09-16T10:33:24.121732+0000","last_clean":"2024-09-16T10:33:24.121732+0000","last_became_active":"2024-09-16T10:33:17.857301+0000","last_became_peered":"2024-09-16T10:33:17.857301+0000","last_unstale":"2024-09-16T10:33:24.121732+0000","last_undegraded":"2024-09-16T10:33:24.121732+0000","last_fullsized":"2024-09-16T10:33:24.121732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:14.989023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403995+0000","last_change":"2024-09-16T10:33:19.869229+0000","last_active":"2024-09-16T10:33:24.403995+0000","last_peered":"2024-09-16T10:33:24.403995+0000","last_clean":"2024-09-16T10:33:24.403995+0000","last_became_active":"2024-09-16T10:33:19.868946+0000","last_became_peered":"2024-09-16T10:33:19.868946+0000","last_unstale":"2024-09-16T10:33:24.403995+0000","last_undegraded":"2024-09-16T10:33:24.403995+0000","last_fullsized":"2024-09-16T10:33:24.403995+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:10.003520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404900+0000","last_change":"2024-09-16T10:33:13.825980+0000","last_active":"2024-09-16T10:33:24.404900+0000","last_peered":"2024-09-16T10:33:24.404900+0000","last_clean":"2024-09-16T10:33:24.404900+0000","last_became_active":"2024-09-16T10:33:13.825733+0000","last_became_peered":"2024-09-16T10:33:13.825733+0000","last_unstale":"2024-09-16T10:33:24.404900+0000","last_undegraded":"2024-09-16T10:33:24.404900+0000","last_fullsized":"2024-09-16T10:33:24.404900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:49.775857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121905+0000","last_change":"2024-09-16T10:33:15.848828+0000","last_active":"2024-09-16T10:33:24.121905+0000","last_peered":"2024-09-16T10:33:24.121905+0000","last_clean":"2024-09-16T10:33:24.121905+0000","last_became_active":"2024-09-16T10:33:15.848528+0000","last_became_peered":"2024-09-16T10:33:15.848528+0000","last_unstale":"2024-09-16T10:33:24.121905+0000","last_undegraded":"2024-09-16T10:33:24.121905+0000","last_fullsized":"2024-09-16T10:33:24.121905+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:14.855054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117754+0000","last_change":"2024-09-16T10:33:17.861296+0000","last_active":"2024-09-16T10:33:24.117754+0000","last_peered":"2024-09-16T10:33:24.117754+0000","last_clean":"2024-09-16T10:33:24.117754+0000","last_became_active":"2024-09-16T10:33:17.861029+0000","last_became_peered":"2024-09-16T10:33:17.861029+0000","last_unstale":"2024-09-16T10:33:24.117754+0000","last_undegraded":"2024-09-16T10:33:24.117754+0000","last_fullsized":"2024-09-16T10:33:24.117754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:41:09.753387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404985+0000","last_change":"2024-09-16T10:33:19.909519+0000","last_active":"2024-09-16T10:33:24.404985+0000","last_peered":"2024-09-16T10:33:24.404985+0000","last_clean":"2024-09-16T10:33:24.404985+0000","last_became_active":"2024-09-16T10:33:19.909186+0000","last_became_peered":"2024-09-16T10:33:19.909186+0000","last_unstale":"2024-09-16T10:33:24.404985+0000","last_undegraded":"2024-09-16T10:33:24.404985+0000","last_fullsized":"2024-09-16T10:33:24.404985+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:22.102551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122407+0000","last_change":"2024-09-16T10:33:15.848821+0000","last_active":"2024-09-16T10:33:24.122407+0000","last_peered":"2024-09-16T10:33:24.122407+0000","last_clean":"2024-09-16T10:33:24.122407+0000","last_became_active":"2024-09-16T10:33:15.848521+0000","last_became_peered":"2024-09-16T10:33:15.848521+0000","last_unstale":"2024-09-16T10:33:24.122407+0000","last_undegraded":"2024-09-16T10:33:24.122407+0000","last_fullsized":"2024-09-16T10:33:24.122407+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:16.660941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117537+0000","last_change":"2024-09-16T10:33:13.831413+0000","last_active":"2024-09-16T10:33:24.117537+0000","last_peered":"2024-09-16T10:33:24.117537+0000","last_clean":"2024-09-16T10:33:24.117537+0000","last_became_active":"2024-09-16T10:33:13.831158+0000","last_became_peered":"2024-09-16T10:33:13.831158+0000","last_unstale":"2024-09-16T10:33:24.117537+0000","last_undegraded":"2024-09-16T10:33:24.117537+0000","last_fullsized":"2024-09-16T10:33:24.117537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:02.031293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200772+0000","last_change":"2024-09-16T10:33:17.862043+0000","last_active":"2024-09-16T10:34:27.200772+0000","last_peered":"2024-09-16T10:34:27.200772+0000","last_clean":"2024-09-16T10:34:27.200772+0000","last_became_active":"2024-09-16T10:33:17.861730+0000","last_became_peered":"2024-09-16T10:33:17.861730+0000","last_unstale":"2024-09-16T10:34:27.200772+0000","last_undegraded":"2024-09-16T10:34:27.200772+0000","last_fullsized":"2024-09-16T10:34:27.200772+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:58.055390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405233+0000","last_change":"2024-09-16T10:33:19.878150+0000","last_active":"2024-09-16T10:33:24.405233+0000","last_peered":"2024-09-16T10:33:24.405233+0000","last_clean":"2024-09-16T10:33:24.405233+0000","last_became_active":"2024-09-16T10:33:19.877676+0000","last_became_peered":"2024-09-16T10:33:19.877676+0000","last_unstale":"2024-09-16T10:33:24.405233+0000","last_undegraded":"2024-09-16T10:33:24.405233+0000","last_fullsized":"2024-09-16T10:33:24.405233+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:32:21.719673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117757+0000","last_change":"2024-09-16T10:33:15.844211+0000","last_active":"2024-09-16T10:33:24.117757+0000","last_peered":"2024-09-16T10:33:24.117757+0000","last_clean":"2024-09-16T10:33:24.117757+0000","last_became_active":"2024-09-16T10:33:15.843639+0000","last_became_peered":"2024-09-16T10:33:15.843639+0000","last_unstale":"2024-09-16T10:33:24.117757+0000","last_undegraded":"2024-09-16T10:33:24.117757+0000","last_fullsized":"2024-09-16T10:33:24.117757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:32.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404795+0000","last_change":"2024-09-16T10:33:13.833644+0000","last_active":"2024-09-16T10:33:24.404795+0000","last_peered":"2024-09-16T10:33:24.404795+0000","last_clean":"2024-09-16T10:33:24.404795+0000","last_became_active":"2024-09-16T10:33:13.833324+0000","last_became_peered":"2024-09-16T10:33:13.833324+0000","last_unstale":"2024-09-16T10:33:24.404795+0000","last_undegraded":"2024-09-16T10:33:24.404795+0000","last_fullsized":"2024-09-16T10:33:24.404795+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:47:36.327924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200889+0000","last_change":"2024-09-16T10:33:17.863472+0000","last_active":"2024-09-16T10:34:27.200889+0000","last_peered":"2024-09-16T10:34:27.200889+0000","last_clean":"2024-09-16T10:34:27.200889+0000","last_became_active":"2024-09-16T10:33:17.863190+0000","last_became_peered":"2024-09-16T10:33:17.863190+0000","last_unstale":"2024-09-16T10:34:27.200889+0000","last_undegraded":"2024-09-16T10:34:27.200889+0000","last_fullsized":"2024-09-16T10:34:27.200889+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:42:54.389460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404601+0000","last_change":"2024-09-16T10:33:19.869175+0000","last_active":"2024-09-16T10:33:24.404601+0000","last_peered":"2024-09-16T10:33:24.404601+0000","last_clean":"2024-09-16T10:33:24.404601+0000","last_became_active":"2024-09-16T10:33:19.868882+0000","last_became_peered":"2024-09-16T10:33:19.868882+0000","last_unstale":"2024-09-16T10:33:24.404601+0000","last_undegraded":"2024-09-16T10:33:24.404601+0000","last_fullsized":"2024-09-16T10:33:24.404601+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:16.217936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120901+0000","last_change":"2024-09-16T10:33:15.840633+0000","last_active":"2024-09-16T10:33:24.120901+0000","last_peered":"2024-09-16T10:33:24.120901+0000","last_clean":"2024-09-16T10:33:24.120901+0000","last_became_active":"2024-09-16T10:33:15.840347+0000","last_became_peered":"2024-09-16T10:33:15.840347+0000","last_unstale":"2024-09-16T10:33:24.120901+0000","last_undegraded":"2024-09-16T10:33:24.120901+0000","last_fullsized":"2024-09-16T10:33:24.120901+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:32.306172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403777+0000","last_change":"2024-09-16T10:33:13.830313+0000","last_active":"2024-09-16T10:33:24.403777+0000","last_peered":"2024-09-16T10:33:24.403777+0000","last_clean":"2024-09-16T10:33:24.403777+0000","last_became_active":"2024-09-16T10:33:13.830053+0000","last_became_peered":"2024-09-16T10:33:13.830053+0000","last_unstale":"2024-09-16T10:33:24.403777+0000","last_undegraded":"2024-09-16T10:33:24.403777+0000","last_fullsized":"2024-09-16T10:33:24.403777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:33.961850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121656+0000","last_change":"2024-09-16T10:33:17.861748+0000","last_active":"2024-09-16T10:33:24.121656+0000","last_peered":"2024-09-16T10:33:24.121656+0000","last_clean":"2024-09-16T10:33:24.121656+0000","last_became_active":"2024-09-16T10:33:17.861448+0000","last_became_peered":"2024-09-16T10:33:17.861448+0000","last_unstale":"2024-09-16T10:33:24.121656+0000","last_undegraded":"2024-09-16T10:33:24.121656+0000","last_fullsized":"2024-09-16T10:33:24.121656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:36.398920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403724+0000","last_change":"2024-09-16T10:33:19.877338+0000","last_active":"2024-09-16T10:33:24.403724+0000","last_peered":"2024-09-16T10:33:24.403724+0000","last_clean":"2024-09-16T10:33:24.403724+0000","last_became_active":"2024-09-16T10:33:19.875625+0000","last_became_peered":"2024-09-16T10:33:19.875625+0000","last_unstale":"2024-09-16T10:33:24.403724+0000","last_undegraded":"2024-09-16T10:33:24.403724+0000","last_fullsized":"2024-09-16T10:33:24.403724+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:40.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404042+0000","last_change":"2024-09-16T10:33:15.847097+0000","last_active":"2024-09-16T10:33:24.404042+0000","last_peered":"2024-09-16T10:33:24.404042+0000","last_clean":"2024-09-16T10:33:24.404042+0000","last_became_active":"2024-09-16T10:33:15.845338+0000","last_became_peered":"2024-09-16T10:33:15.845338+0000","last_unstale":"2024-09-16T10:33:24.404042+0000","last_undegraded":"2024-09-16T10:33:24.404042+0000","last_fullsized":"2024-09-16T10:33:24.404042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:19.356069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121935+0000","last_change":"2024-09-16T10:33:13.826300+0000","last_active":"2024-09-16T10:33:24.121935+0000","last_peered":"2024-09-16T10:33:24.121935+0000","last_clean":"2024-09-16T10:33:24.121935+0000","last_became_active":"2024-09-16T10:33:13.826015+0000","last_became_peered":"2024-09-16T10:33:13.826015+0000","last_unstale":"2024-09-16T10:33:24.121935+0000","last_undegraded":"2024-09-16T10:33:24.121935+0000","last_fullsized":"2024-09-16T10:33:24.121935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:57.183144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120726+0000","last_change":"2024-09-16T10:33:17.861980+0000","last_active":"2024-09-16T10:33:24.120726+0000","last_peered":"2024-09-16T10:33:24.120726+0000","last_clean":"2024-09-16T10:33:24.120726+0000","last_became_active":"2024-09-16T10:33:17.861704+0000","last_became_peered":"2024-09-16T10:33:17.861704+0000","last_unstale":"2024-09-16T10:33:24.120726+0000","last_undegraded":"2024-09-16T10:33:24.120726+0000","last_fullsized":"2024-09-16T10:33:24.120726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:55.118484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120763+0000","last_change":"2024-09-16T10:33:19.877272+0000","last_active":"2024-09-16T10:33:24.120763+0000","last_peered":"2024-09-16T10:33:24.120763+0000","last_clean":"2024-09-16T10:33:24.120763+0000","last_became_active":"2024-09-16T10:33:19.876986+0000","last_became_peered":"2024-09-16T10:33:19.876986+0000","last_unstale":"2024-09-16T10:33:24.120763+0000","last_undegraded":"2024-09-16T10:33:24.120763+0000","last_fullsized":"2024-09-16T10:33:24.120763+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:44:40.314655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121463+0000","last_change":"2024-09-16T10:33:15.848107+0000","last_active":"2024-09-16T10:33:24.121463+0000","last_peered":"2024-09-16T10:33:24.121463+0000","last_clean":"2024-09-16T10:33:24.121463+0000","last_became_active":"2024-09-16T10:33:15.847695+0000","last_became_peered":"2024-09-16T10:33:15.847695+0000","last_unstale":"2024-09-16T10:33:24.121463+0000","last_undegraded":"2024-09-16T10:33:24.121463+0000","last_fullsized":"2024-09-16T10:33:24.121463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:27:51.896174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117814+0000","last_change":"2024-09-16T10:33:13.828318+0000","last_active":"2024-09-16T10:33:24.117814+0000","last_peered":"2024-09-16T10:33:24.117814+0000","last_clean":"2024-09-16T10:33:24.117814+0000","last_became_active":"2024-09-16T10:33:13.828064+0000","last_became_peered":"2024-09-16T10:33:13.828064+0000","last_unstale":"2024-09-16T10:33:24.117814+0000","last_undegraded":"2024-09-16T10:33:24.117814+0000","last_fullsized":"2024-09-16T10:33:24.117814+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:38.053804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404153+0000","last_change":"2024-09-16T10:33:17.862953+0000","last_active":"2024-09-16T10:33:24.404153+0000","last_peered":"2024-09-16T10:33:24.404153+0000","last_clean":"2024-09-16T10:33:24.404153+0000","last_became_active":"2024-09-16T10:33:17.862706+0000","last_became_peered":"2024-09-16T10:33:17.862706+0000","last_unstale":"2024-09-16T10:33:24.404153+0000","last_undegraded":"2024-09-16T10:33:24.404153+0000","last_fullsized":"2024-09-16T10:33:24.404153+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:13.727989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403643+0000","last_change":"2024-09-16T10:33:19.870777+0000","last_active":"2024-09-16T10:33:24.403643+0000","last_peered":"2024-09-16T10:33:24.403643+0000","last_clean":"2024-09-16T10:33:24.403643+0000","last_became_active":"2024-09-16T10:33:19.870479+0000","last_became_peered":"2024-09-16T10:33:19.870479+0000","last_unstale":"2024-09-16T10:33:24.403643+0000","last_undegraded":"2024-09-16T10:33:24.403643+0000","last_fullsized":"2024-09-16T10:33:24.403643+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:04.103959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120983+0000","last_change":"2024-09-16T10:33:15.848126+0000","last_active":"2024-09-16T10:33:24.120983+0000","last_peered":"2024-09-16T10:33:24.120983+0000","last_clean":"2024-09-16T10:33:24.120983+0000","last_became_active":"2024-09-16T10:33:15.847882+0000","last_became_peered":"2024-09-16T10:33:15.847882+0000","last_unstale":"2024-09-16T10:33:24.120983+0000","last_undegraded":"2024-09-16T10:33:24.120983+0000","last_fullsized":"2024-09-16T10:33:24.120983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:57.050068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404154+0000","last_change":"2024-09-16T10:33:13.824934+0000","last_active":"2024-09-16T10:33:24.404154+0000","last_peered":"2024-09-16T10:33:24.404154+0000","last_clean":"2024-09-16T10:33:24.404154+0000","last_became_active":"2024-09-16T10:33:13.824667+0000","last_became_peered":"2024-09-16T10:33:13.824667+0000","last_unstale":"2024-09-16T10:33:24.404154+0000","last_undegraded":"2024-09-16T10:33:24.404154+0000","last_fullsized":"2024-09-16T10:33:24.404154+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:56.178015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405376+0000","last_change":"2024-09-16T10:33:17.860036+0000","last_active":"2024-09-16T10:33:24.405376+0000","last_peered":"2024-09-16T10:33:24.405376+0000","last_clean":"2024-09-16T10:33:24.405376+0000","last_became_active":"2024-09-16T10:33:17.859392+0000","last_became_peered":"2024-09-16T10:33:17.859392+0000","last_unstale":"2024-09-16T10:33:24.405376+0000","last_undegraded":"2024-09-16T10:33:24.405376+0000","last_fullsized":"2024-09-16T10:33:24.405376+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:17.305923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121492+0000","last_change":"2024-09-16T10:33:19.909507+0000","last_active":"2024-09-16T10:33:24.121492+0000","last_peered":"2024-09-16T10:33:24.121492+0000","last_clean":"2024-09-16T10:33:24.121492+0000","last_became_active":"2024-09-16T10:33:19.909204+0000","last_became_peered":"2024-09-16T10:33:19.909204+0000","last_unstale":"2024-09-16T10:33:24.121492+0000","last_undegraded":"2024-09-16T10:33:24.121492+0000","last_fullsized":"2024-09-16T10:33:24.121492+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:44.376563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403866+0000","last_change":"2024-09-16T10:33:19.872836+0000","last_active":"2024-09-16T10:33:24.403866+0000","last_peered":"2024-09-16T10:33:24.403866+0000","last_clean":"2024-09-16T10:33:24.403866+0000","last_became_active":"2024-09-16T10:33:19.872554+0000","last_became_peered":"2024-09-16T10:33:19.872554+0000","last_unstale":"2024-09-16T10:33:24.403866+0000","last_undegraded":"2024-09-16T10:33:24.403866+0000","last_fullsized":"2024-09-16T10:33:24.403866+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:08:13.249106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117246+0000","last_change":"2024-09-16T10:33:13.831366+0000","last_active":"2024-09-16T10:33:24.117246+0000","last_peered":"2024-09-16T10:33:24.117246+0000","last_clean":"2024-09-16T10:33:24.117246+0000","last_became_active":"2024-09-16T10:33:13.831045+0000","last_became_peered":"2024-09-16T10:33:13.831045+0000","last_unstale":"2024-09-16T10:33:24.117246+0000","last_undegraded":"2024-09-16T10:33:24.117246+0000","last_fullsized":"2024-09-16T10:33:24.117246+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:11.658752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405298+0000","last_change":"2024-09-16T10:33:15.854039+0000","last_active":"2024-09-16T10:33:24.405298+0000","last_peered":"2024-09-16T10:33:24.405298+0000","last_clean":"2024-09-16T10:33:24.405298+0000","last_became_active":"2024-09-16T10:33:15.853711+0000","last_became_peered":"2024-09-16T10:33:15.853711+0000","last_unstale":"2024-09-16T10:33:24.405298+0000","last_undegraded":"2024-09-16T10:33:24.405298+0000","last_fullsized":"2024-09-16T10:33:24.405298+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:26.060755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200733+0000","last_change":"2024-09-16T10:33:17.866228+0000","last_active":"2024-09-16T10:34:27.200733+0000","last_peered":"2024-09-16T10:34:27.200733+0000","last_clean":"2024-09-16T10:34:27.200733+0000","last_became_active":"2024-09-16T10:33:17.865902+0000","last_became_peered":"2024-09-16T10:33:17.865902+0000","last_unstale":"2024-09-16T10:34:27.200733+0000","last_undegraded":"2024-09-16T10:34:27.200733+0000","last_fullsized":"2024-09-16T10:34:27.200733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:26.959193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120627+0000","last_change":"2024-09-16T10:33:19.870399+0000","last_active":"2024-09-16T10:33:24.120627+0000","last_peered":"2024-09-16T10:33:24.120627+0000","last_clean":"2024-09-16T10:33:24.120627+0000","last_became_active":"2024-09-16T10:33:19.870110+0000","last_became_peered":"2024-09-16T10:33:19.870110+0000","last_unstale":"2024-09-16T10:33:24.120627+0000","last_undegraded":"2024-09-16T10:33:24.120627+0000","last_fullsized":"2024-09-16T10:33:24.120627+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:53:10.134858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121303+0000","last_change":"2024-09-16T10:33:13.831459+0000","last_active":"2024-09-16T10:33:24.121303+0000","last_peered":"2024-09-16T10:33:24.121303+0000","last_clean":"2024-09-16T10:33:24.121303+0000","last_became_active":"2024-09-16T10:33:13.831054+0000","last_became_peered":"2024-09-16T10:33:13.831054+0000","last_unstale":"2024-09-16T10:33:24.121303+0000","last_undegraded":"2024-09-16T10:33:24.121303+0000","last_fullsized":"2024-09-16T10:33:24.121303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:26.290480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404282+0000","last_change":"2024-09-16T10:33:15.843213+0000","last_active":"2024-09-16T10:33:24.404282+0000","last_peered":"2024-09-16T10:33:24.404282+0000","last_clean":"2024-09-16T10:33:24.404282+0000","last_became_active":"2024-09-16T10:33:15.842893+0000","last_became_peered":"2024-09-16T10:33:15.842893+0000","last_unstale":"2024-09-16T10:33:24.404282+0000","last_undegraded":"2024-09-16T10:33:24.404282+0000","last_fullsized":"2024-09-16T10:33:24.404282+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:48.208043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405295+0000","last_change":"2024-09-16T10:33:17.858923+0000","last_active":"2024-09-16T10:33:24.405295+0000","last_peered":"2024-09-16T10:33:24.405295+0000","last_clean":"2024-09-16T10:33:24.405295+0000","last_became_active":"2024-09-16T10:33:17.858605+0000","last_became_peered":"2024-09-16T10:33:17.858605+0000","last_unstale":"2024-09-16T10:33:24.405295+0000","last_undegraded":"2024-09-16T10:33:24.405295+0000","last_fullsized":"2024-09-16T10:33:24.405295+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:10:21.376810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405106+0000","last_change":"2024-09-16T10:33:19.908026+0000","last_active":"2024-09-16T10:33:24.405106+0000","last_peered":"2024-09-16T10:33:24.405106+0000","last_clean":"2024-09-16T10:33:24.405106+0000","last_became_active":"2024-09-16T10:33:19.907758+0000","last_became_peered":"2024-09-16T10:33:19.907758+0000","last_unstale":"2024-09-16T10:33:24.405106+0000","last_undegraded":"2024-09-16T10:33:24.405106+0000","last_fullsized":"2024-09-16T10:33:24.405106+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:20.958848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405422+0000","last_change":"2024-09-16T10:33:13.820521+0000","last_active":"2024-09-16T10:33:24.405422+0000","last_peered":"2024-09-16T10:33:24.405422+0000","last_clean":"2024-09-16T10:33:24.405422+0000","last_became_active":"2024-09-16T10:33:13.820193+0000","last_became_peered":"2024-09-16T10:33:13.820193+0000","last_unstale":"2024-09-16T10:33:24.405422+0000","last_undegraded":"2024-09-16T10:33:24.405422+0000","last_fullsized":"2024-09-16T10:33:24.405422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:50.955174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404487+0000","last_change":"2024-09-16T10:33:15.842037+0000","last_active":"2024-09-16T10:33:24.404487+0000","last_peered":"2024-09-16T10:33:24.404487+0000","last_clean":"2024-09-16T10:33:24.404487+0000","last_became_active":"2024-09-16T10:33:15.841722+0000","last_became_peered":"2024-09-16T10:33:15.841722+0000","last_unstale":"2024-09-16T10:33:24.404487+0000","last_undegraded":"2024-09-16T10:33:24.404487+0000","last_fullsized":"2024-09-16T10:33:24.404487+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:27.128996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121778+0000","last_change":"2024-09-16T10:33:17.859305+0000","last_active":"2024-09-16T10:33:24.121778+0000","last_peered":"2024-09-16T10:33:24.121778+0000","last_clean":"2024-09-16T10:33:24.121778+0000","last_became_active":"2024-09-16T10:33:17.858981+0000","last_became_peered":"2024-09-16T10:33:17.858981+0000","last_unstale":"2024-09-16T10:33:24.121778+0000","last_undegraded":"2024-09-16T10:33:24.121778+0000","last_fullsized":"2024-09-16T10:33:24.121778+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:10.081563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121653+0000","last_change":"2024-09-16T10:33:19.908120+0000","last_active":"2024-09-16T10:33:24.121653+0000","last_peered":"2024-09-16T10:33:24.121653+0000","last_clean":"2024-09-16T10:33:24.121653+0000","last_became_active":"2024-09-16T10:33:19.907807+0000","last_became_peered":"2024-09-16T10:33:19.907807+0000","last_unstale":"2024-09-16T10:33:24.121653+0000","last_undegraded":"2024-09-16T10:33:24.121653+0000","last_fullsized":"2024-09-16T10:33:24.121653+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:23.684854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122075+0000","last_change":"2024-09-16T10:33:13.823143+0000","last_active":"2024-09-16T10:33:24.122075+0000","last_peered":"2024-09-16T10:33:24.122075+0000","last_clean":"2024-09-16T10:33:24.122075+0000","last_became_active":"2024-09-16T10:33:13.822870+0000","last_became_peered":"2024-09-16T10:33:13.822870+0000","last_unstale":"2024-09-16T10:33:24.122075+0000","last_undegraded":"2024-09-16T10:33:24.122075+0000","last_fullsized":"2024-09-16T10:33:24.122075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:46.899315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":40,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122510+0000","last_change":"2024-09-16T10:33:15.848076+0000","last_active":"2024-09-16T10:33:24.122510+0000","last_peered":"2024-09-16T10:33:24.122510+0000","last_clean":"2024-09-16T10:33:24.122510+0000","last_became_active":"2024-09-16T10:33:15.847791+0000","last_became_peered":"2024-09-16T10:33:15.847791+0000","last_unstale":"2024-09-16T10:33:24.122510+0000","last_undegraded":"2024-09-16T10:33:24.122510+0000","last_fullsized":"2024-09-16T10:33:24.122510+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:17.088207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403950+0000","last_change":"2024-09-16T10:33:17.865501+0000","last_active":"2024-09-16T10:33:24.403950+0000","last_peered":"2024-09-16T10:33:24.403950+0000","last_clean":"2024-09-16T10:33:24.403950+0000","last_became_active":"2024-09-16T10:33:17.865169+0000","last_became_peered":"2024-09-16T10:33:17.865169+0000","last_unstale":"2024-09-16T10:33:24.403950+0000","last_undegraded":"2024-09-16T10:33:24.403950+0000","last_fullsized":"2024-09-16T10:33:24.403950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:23.752283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201303,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32452,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708604,"statfs":{"total":95990841344,"available":95957610496,"internally_reserved":0,"allocated":1372160,"data_stored":911333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.505}]},{"osd":1,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47999999999999998}]},{"osd":2,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.495},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57399999999999995}]},{"osd":3,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54700000000000004}]},{"osd":4,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.377}]},{"osd":6,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.443},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.45400000000000001}]}]},{"osd":6,"up_from":47,"seq":201863462940,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32428,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708628,"statfs":{"total":95990841344,"available":95957635072,"internally_reserved":0,"allocated":1347584,"data_stored":909551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.621},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.53500000000000003}]},{"osd":1,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.59399999999999997}]},{"osd":3,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.35899999999999999}]},{"osd":5,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.376},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.442}]},{"osd":7,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.505}]}]},{"osd":5,"up_from":39,"seq":167503724576,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713292,"statfs":{"total":95990841344,"available":95962411008,"internally_reserved":0,"allocated":761856,"data_stored":319397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.503,"15min":0.499},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.66200000000000003}]},{"osd":1,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.48999999999999999}]},{"osd":2,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63800000000000001}]},{"osd":3,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79500000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.503}]},{"osd":4,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38200000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.29799999999999999}]},{"osd":6,"last update":"Mon Sep 16 10:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.375}]},{"osd":7,"last update":"Mon Sep 16 10:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.36099999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713280,"statfs":{"total":95990841344,"available":95962398720,"internally_reserved":0,"allocated":774144,"data_stored":319522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.28499999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.59899999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.313},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.38500000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60699999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.29299999999999998}]},{"osd":1,"last update":"Mon Sep 16 10:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.56699999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.65100000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.35299999999999998}]},{"osd":2,"last update":"Mon Sep 16 10:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52600000000000002,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61499999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.52500000000000002,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.622,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.378}]},{"osd":3,"last update":"Mon Sep 16 10:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66200000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Mon Sep 16 10:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.42599999999999999,"15min":0.41599999999999998},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66800000000000004,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.49099999999999999}]},{"osd":6,"last update":"Mon Sep 16 10:33:36 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.46999999999999997}]},{"osd":7,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.53800000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713256,"statfs":{"total":95990841344,"available":95962374144,"internally_reserved":0,"allocated":798720,"data_stored":320364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53700000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.434,"15min":0.44500000000000001},"min":{"1min":0.32700000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.48299999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.50800000000000001}]},{"osd":1,"last update":"Mon Sep 16 10:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55800000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41699999999999998,"15min":0.42299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69399999999999995,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.432}]},{"osd":2,"last update":"Mon Sep 16 10:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42799999999999999,"15min":0.434},"min":{"1min":0.29999999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58599999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.46600000000000003,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57499999999999996,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.45100000000000001}]},{"osd":4,"last update":"Mon Sep 16 10:33:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Sep 16 10:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90100000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57399999999999995}]},{"osd":6,"last update":"Mon Sep 16 10:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.628}]},{"osd":7,"last update":"Mon Sep 16 10:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60099999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713292,"statfs":{"total":95990841344,"available":95962411008,"internally_reserved":0,"allocated":761856,"data_stored":320263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35699999999999998,"15min":0.36099999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.502,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.379},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.374,"15min":0.379},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.27100000000000002}]},{"osd":1,"last update":"Mon Sep 16 10:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.377,"15min":0.378},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52800000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52400000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.40999999999999998}]},{"osd":3,"last update":"Mon Sep 16 10:34:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.41199999999999998,"15min":0.42799999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53800000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.441,"15min":0.45500000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59599999999999997,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.32900000000000001}]},{"osd":4,"last update":"Mon Sep 16 10:33:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.38,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.39100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75900000000000001,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.69099999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Sep 16 10:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73999999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71099999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.5}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713240,"statfs":{"total":95990841344,"available":95962357760,"internally_reserved":0,"allocated":815104,"data_stored":321628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.316},"min":{"1min":0.28199999999999997,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.47999999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.33700000000000002,"15min":0.316},"min":{"1min":0.28999999999999998,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.307}]},{"osd":2,"last update":"Mon Sep 16 10:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33100000000000002,"15min":0.32500000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.374,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50900000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.35599999999999998}]},{"osd":3,"last update":"Mon Sep 16 10:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42299999999999999,"15min":0.42899999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.56299999999999994,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.42299999999999999},"min":{"1min":0.27800000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59499999999999997,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.434}]},{"osd":4,"last update":"Mon Sep 16 10:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71299999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40500000000000003}]},{"osd":5,"last update":"Mon Sep 16 10:34:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54100000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.42899999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68899999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41999999999999998}]},{"osd":6,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46300000000000002}]},{"osd":7,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.374}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32452,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708604,"statfs":{"total":95990841344,"available":95957610496,"internally_reserved":0,"allocated":1372160,"data_stored":910941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 10:33:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.34899999999999998,"15min":0.34300000000000003},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.38100000000000001,"15min":0.36499999999999999},"min":{"1min":0.27500000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.373}]},{"osd":2,"last update":"Mon Sep 16 10:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.51800000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36099999999999999}]},{"osd":3,"last update":"Mon Sep 16 10:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.437,"15min":0.442},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.495,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.316}]},{"osd":4,"last update":"Mon Sep 16 10:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.38100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.51100000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.73299999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56200000000000006}]},{"osd":5,"last update":"Mon Sep 16 10:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.46400000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54000000000000004}]},{"osd":6,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.40300000000000002}]},{"osd":7,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.42999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T10:34:36.602 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T10:34:36.801 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:37.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:37 smithi164 ceph-mon[40346]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T10:34:37.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:37 smithi164 ceph-mon[25828]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T10:34:37.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:37 smithi175 ceph-mon[30341]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T10:34:38.259 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:38 smithi164 ceph-mon[40346]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:38.259 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:38 smithi164 ceph-mon[25828]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:38.425 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:38 smithi175 ceph-mon[30341]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:38.629 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:34:38.631 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-16T10:34:39.426 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-16T10:34:37.005560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596246,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1026,"num_read_kb":869,"num_write":675,"num_write_kb":4025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":657,"ondisk_log_size":657,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":236252,"kb_used_data":7816,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":749692196,"statfs":{"total":767926730752,"available":767684808704,"internally_reserved":0,"allocated":8003584,"data_stored":4332999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":233819704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003078"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121395+0000","last_change":"2024-09-16T10:33:19.909495+0000","last_active":"2024-09-16T10:33:24.121395+0000","last_peered":"2024-09-16T10:33:24.121395+0000","last_clean":"2024-09-16T10:33:24.121395+0000","last_became_active":"2024-09-16T10:33:19.909182+0000","last_became_peered":"2024-09-16T10:33:19.909182+0000","last_unstale":"2024-09-16T10:33:24.121395+0000","last_undegraded":"2024-09-16T10:33:24.121395+0000","last_fullsized":"2024-09-16T10:33:24.121395+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:58.791582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405015+0000","last_change":"2024-09-16T10:33:13.833129+0000","last_active":"2024-09-16T10:33:24.405015+0000","last_peered":"2024-09-16T10:33:24.405015+0000","last_clean":"2024-09-16T10:33:24.405015+0000","last_became_active":"2024-09-16T10:33:13.832647+0000","last_became_peered":"2024-09-16T10:33:13.832647+0000","last_unstale":"2024-09-16T10:33:24.405015+0000","last_undegraded":"2024-09-16T10:33:24.405015+0000","last_fullsized":"2024-09-16T10:33:24.405015+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:37.041695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121171+0000","last_change":"2024-09-16T10:33:15.846271+0000","last_active":"2024-09-16T10:33:24.121171+0000","last_peered":"2024-09-16T10:33:24.121171+0000","last_clean":"2024-09-16T10:33:24.121171+0000","last_became_active":"2024-09-16T10:33:15.845997+0000","last_became_peered":"2024-09-16T10:33:15.845997+0000","last_unstale":"2024-09-16T10:33:24.121171+0000","last_undegraded":"2024-09-16T10:33:24.121171+0000","last_fullsized":"2024-09-16T10:33:24.121171+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:18:51.246178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403491+0000","last_change":"2024-09-16T10:33:17.861562+0000","last_active":"2024-09-16T10:33:24.403491+0000","last_peered":"2024-09-16T10:33:24.403491+0000","last_clean":"2024-09-16T10:33:24.403491+0000","last_became_active":"2024-09-16T10:33:17.861165+0000","last_became_peered":"2024-09-16T10:33:17.861165+0000","last_unstale":"2024-09-16T10:33:24.403491+0000","last_undegraded":"2024-09-16T10:33:24.403491+0000","last_fullsized":"2024-09-16T10:33:24.403491+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:18.765194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122371+0000","last_change":"2024-09-16T10:33:13.830466+0000","last_active":"2024-09-16T10:33:24.122371+0000","last_peered":"2024-09-16T10:33:24.122371+0000","last_clean":"2024-09-16T10:33:24.122371+0000","last_became_active":"2024-09-16T10:33:13.830209+0000","last_became_peered":"2024-09-16T10:33:13.830209+0000","last_unstale":"2024-09-16T10:33:24.122371+0000","last_undegraded":"2024-09-16T10:33:24.122371+0000","last_fullsized":"2024-09-16T10:33:24.122371+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:51.270208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403574+0000","last_change":"2024-09-16T10:33:15.848110+0000","last_active":"2024-09-16T10:33:24.403574+0000","last_peered":"2024-09-16T10:33:24.403574+0000","last_clean":"2024-09-16T10:33:24.403574+0000","last_became_active":"2024-09-16T10:33:15.847813+0000","last_became_peered":"2024-09-16T10:33:15.847813+0000","last_unstale":"2024-09-16T10:33:24.403574+0000","last_undegraded":"2024-09-16T10:33:24.403574+0000","last_fullsized":"2024-09-16T10:33:24.403574+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:27.473578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120565+0000","last_change":"2024-09-16T10:33:17.858479+0000","last_active":"2024-09-16T10:33:24.120565+0000","last_peered":"2024-09-16T10:33:24.120565+0000","last_clean":"2024-09-16T10:33:24.120565+0000","last_became_active":"2024-09-16T10:33:17.858183+0000","last_became_peered":"2024-09-16T10:33:17.858183+0000","last_unstale":"2024-09-16T10:33:24.120565+0000","last_undegraded":"2024-09-16T10:33:24.120565+0000","last_fullsized":"2024-09-16T10:33:24.120565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:24.087752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403567+0000","last_change":"2024-09-16T10:33:19.869599+0000","last_active":"2024-09-16T10:33:24.403567+0000","last_peered":"2024-09-16T10:33:24.403567+0000","last_clean":"2024-09-16T10:33:24.403567+0000","last_became_active":"2024-09-16T10:33:19.869045+0000","last_became_peered":"2024-09-16T10:33:19.869045+0000","last_unstale":"2024-09-16T10:33:24.403567+0000","last_undegraded":"2024-09-16T10:33:24.403567+0000","last_fullsized":"2024-09-16T10:33:24.403567+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:50:10.557168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405282+0000","last_change":"2024-09-16T10:33:13.830662+0000","last_active":"2024-09-16T10:33:24.405282+0000","last_peered":"2024-09-16T10:33:24.405282+0000","last_clean":"2024-09-16T10:33:24.405282+0000","last_became_active":"2024-09-16T10:33:13.830216+0000","last_became_peered":"2024-09-16T10:33:13.830216+0000","last_unstale":"2024-09-16T10:33:24.405282+0000","last_undegraded":"2024-09-16T10:33:24.405282+0000","last_fullsized":"2024-09-16T10:33:24.405282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:01:37.779851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":27,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403555+0000","last_change":"2024-09-16T10:33:15.848652+0000","last_active":"2024-09-16T10:33:24.403555+0000","last_peered":"2024-09-16T10:33:24.403555+0000","last_clean":"2024-09-16T10:33:24.403555+0000","last_became_active":"2024-09-16T10:33:15.848338+0000","last_became_peered":"2024-09-16T10:33:15.848338+0000","last_unstale":"2024-09-16T10:33:24.403555+0000","last_undegraded":"2024-09-16T10:33:24.403555+0000","last_fullsized":"2024-09-16T10:33:24.403555+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:51.768572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403718+0000","last_change":"2024-09-16T10:33:17.859749+0000","last_active":"2024-09-16T10:33:24.403718+0000","last_peered":"2024-09-16T10:33:24.403718+0000","last_clean":"2024-09-16T10:33:24.403718+0000","last_became_active":"2024-09-16T10:33:17.859335+0000","last_became_peered":"2024-09-16T10:33:17.859335+0000","last_unstale":"2024-09-16T10:33:24.403718+0000","last_undegraded":"2024-09-16T10:33:24.403718+0000","last_fullsized":"2024-09-16T10:33:24.403718+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:47.684045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117298+0000","last_change":"2024-09-16T10:33:19.873926+0000","last_active":"2024-09-16T10:33:24.117298+0000","last_peered":"2024-09-16T10:33:24.117298+0000","last_clean":"2024-09-16T10:33:24.117298+0000","last_became_active":"2024-09-16T10:33:19.873588+0000","last_became_peered":"2024-09-16T10:33:19.873588+0000","last_unstale":"2024-09-16T10:33:24.117298+0000","last_undegraded":"2024-09-16T10:33:24.117298+0000","last_fullsized":"2024-09-16T10:33:24.117298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:31.453260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405282+0000","last_change":"2024-09-16T10:33:13.825016+0000","last_active":"2024-09-16T10:33:24.405282+0000","last_peered":"2024-09-16T10:33:24.405282+0000","last_clean":"2024-09-16T10:33:24.405282+0000","last_became_active":"2024-09-16T10:33:13.824739+0000","last_became_peered":"2024-09-16T10:33:13.824739+0000","last_unstale":"2024-09-16T10:33:24.405282+0000","last_undegraded":"2024-09-16T10:33:24.405282+0000","last_fullsized":"2024-09-16T10:33:24.405282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:46.420520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403760+0000","last_change":"2024-09-16T10:33:15.853868+0000","last_active":"2024-09-16T10:33:24.403760+0000","last_peered":"2024-09-16T10:33:24.403760+0000","last_clean":"2024-09-16T10:33:24.403760+0000","last_became_active":"2024-09-16T10:33:15.853552+0000","last_became_peered":"2024-09-16T10:33:15.853552+0000","last_unstale":"2024-09-16T10:33:24.403760+0000","last_undegraded":"2024-09-16T10:33:24.403760+0000","last_fullsized":"2024-09-16T10:33:24.403760+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:31.151079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117384+0000","last_change":"2024-09-16T10:33:17.857859+0000","last_active":"2024-09-16T10:33:24.117384+0000","last_peered":"2024-09-16T10:33:24.117384+0000","last_clean":"2024-09-16T10:33:24.117384+0000","last_became_active":"2024-09-16T10:33:17.857526+0000","last_became_peered":"2024-09-16T10:33:17.857526+0000","last_unstale":"2024-09-16T10:33:24.117384+0000","last_undegraded":"2024-09-16T10:33:24.117384+0000","last_fullsized":"2024-09-16T10:33:24.117384+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:07.015810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403877+0000","last_change":"2024-09-16T10:33:19.876901+0000","last_active":"2024-09-16T10:33:24.403877+0000","last_peered":"2024-09-16T10:33:24.403877+0000","last_clean":"2024-09-16T10:33:24.403877+0000","last_became_active":"2024-09-16T10:33:19.875179+0000","last_became_peered":"2024-09-16T10:33:19.875179+0000","last_unstale":"2024-09-16T10:33:24.403877+0000","last_undegraded":"2024-09-16T10:33:24.403877+0000","last_fullsized":"2024-09-16T10:33:24.403877+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:12.257683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":45,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405219+0000","last_change":"2024-09-16T10:33:15.845978+0000","last_active":"2024-09-16T10:33:24.405219+0000","last_peered":"2024-09-16T10:33:24.405219+0000","last_clean":"2024-09-16T10:33:24.405219+0000","last_became_active":"2024-09-16T10:33:15.845619+0000","last_became_peered":"2024-09-16T10:33:15.845619+0000","last_unstale":"2024-09-16T10:33:24.405219+0000","last_undegraded":"2024-09-16T10:33:24.405219+0000","last_fullsized":"2024-09-16T10:33:24.405219+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:24:07.328422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404998+0000","last_change":"2024-09-16T10:33:13.825242+0000","last_active":"2024-09-16T10:33:24.404998+0000","last_peered":"2024-09-16T10:33:24.404998+0000","last_clean":"2024-09-16T10:33:24.404998+0000","last_became_active":"2024-09-16T10:33:13.824994+0000","last_became_peered":"2024-09-16T10:33:13.824994+0000","last_unstale":"2024-09-16T10:33:24.404998+0000","last_undegraded":"2024-09-16T10:33:24.404998+0000","last_fullsized":"2024-09-16T10:33:24.404998+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:33.720530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121078+0000","last_change":"2024-09-16T10:33:17.857808+0000","last_active":"2024-09-16T10:33:24.121078+0000","last_peered":"2024-09-16T10:33:24.121078+0000","last_clean":"2024-09-16T10:33:24.121078+0000","last_became_active":"2024-09-16T10:33:17.857513+0000","last_became_peered":"2024-09-16T10:33:17.857513+0000","last_unstale":"2024-09-16T10:33:24.121078+0000","last_undegraded":"2024-09-16T10:33:24.121078+0000","last_fullsized":"2024-09-16T10:33:24.121078+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:10.733400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405141+0000","last_change":"2024-09-16T10:33:19.870176+0000","last_active":"2024-09-16T10:33:24.405141+0000","last_peered":"2024-09-16T10:33:24.405141+0000","last_clean":"2024-09-16T10:33:24.405141+0000","last_became_active":"2024-09-16T10:33:19.869907+0000","last_became_peered":"2024-09-16T10:33:19.869907+0000","last_unstale":"2024-09-16T10:33:24.405141+0000","last_undegraded":"2024-09-16T10:33:24.405141+0000","last_fullsized":"2024-09-16T10:33:24.405141+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:31:33.611070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404853+0000","last_change":"2024-09-16T10:33:15.853956+0000","last_active":"2024-09-16T10:33:24.404853+0000","last_peered":"2024-09-16T10:33:24.404853+0000","last_clean":"2024-09-16T10:33:24.404853+0000","last_became_active":"2024-09-16T10:33:15.853626+0000","last_became_peered":"2024-09-16T10:33:15.853626+0000","last_unstale":"2024-09-16T10:33:24.404853+0000","last_undegraded":"2024-09-16T10:33:24.404853+0000","last_fullsized":"2024-09-16T10:33:24.404853+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:51:44.841430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121127+0000","last_change":"2024-09-16T10:33:13.833976+0000","last_active":"2024-09-16T10:33:24.121127+0000","last_peered":"2024-09-16T10:33:24.121127+0000","last_clean":"2024-09-16T10:33:24.121127+0000","last_became_active":"2024-09-16T10:33:13.833041+0000","last_became_peered":"2024-09-16T10:33:13.833041+0000","last_unstale":"2024-09-16T10:33:24.121127+0000","last_undegraded":"2024-09-16T10:33:24.121127+0000","last_fullsized":"2024-09-16T10:33:24.121127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:56.080692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200626+0000","last_change":"2024-09-16T10:33:17.857392+0000","last_active":"2024-09-16T10:34:27.200626+0000","last_peered":"2024-09-16T10:34:27.200626+0000","last_clean":"2024-09-16T10:34:27.200626+0000","last_became_active":"2024-09-16T10:33:17.857052+0000","last_became_peered":"2024-09-16T10:33:17.857052+0000","last_unstale":"2024-09-16T10:34:27.200626+0000","last_undegraded":"2024-09-16T10:34:27.200626+0000","last_fullsized":"2024-09-16T10:34:27.200626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:44:19.221848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405032+0000","last_change":"2024-09-16T10:33:19.866574+0000","last_active":"2024-09-16T10:33:24.405032+0000","last_peered":"2024-09-16T10:33:24.405032+0000","last_clean":"2024-09-16T10:33:24.405032+0000","last_became_active":"2024-09-16T10:33:19.866300+0000","last_became_peered":"2024-09-16T10:33:19.866300+0000","last_unstale":"2024-09-16T10:33:24.405032+0000","last_undegraded":"2024-09-16T10:33:24.405032+0000","last_fullsized":"2024-09-16T10:33:24.405032+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:28:46.742763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122075+0000","last_change":"2024-09-16T10:33:15.852171+0000","last_active":"2024-09-16T10:33:24.122075+0000","last_peered":"2024-09-16T10:33:24.122075+0000","last_clean":"2024-09-16T10:33:24.122075+0000","last_became_active":"2024-09-16T10:33:15.850300+0000","last_became_peered":"2024-09-16T10:33:15.850300+0000","last_unstale":"2024-09-16T10:33:24.122075+0000","last_undegraded":"2024-09-16T10:33:24.122075+0000","last_fullsized":"2024-09-16T10:33:24.122075+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:32.967386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120794+0000","last_change":"2024-09-16T10:33:13.834432+0000","last_active":"2024-09-16T10:33:24.120794+0000","last_peered":"2024-09-16T10:33:24.120794+0000","last_clean":"2024-09-16T10:33:24.120794+0000","last_became_active":"2024-09-16T10:33:13.833975+0000","last_became_peered":"2024-09-16T10:33:13.833975+0000","last_unstale":"2024-09-16T10:33:24.120794+0000","last_undegraded":"2024-09-16T10:33:24.120794+0000","last_fullsized":"2024-09-16T10:33:24.120794+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:55.639735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200708+0000","last_change":"2024-09-16T10:33:17.858688+0000","last_active":"2024-09-16T10:34:27.200708+0000","last_peered":"2024-09-16T10:34:27.200708+0000","last_clean":"2024-09-16T10:34:27.200708+0000","last_became_active":"2024-09-16T10:33:17.858326+0000","last_became_peered":"2024-09-16T10:33:17.858326+0000","last_unstale":"2024-09-16T10:34:27.200708+0000","last_undegraded":"2024-09-16T10:34:27.200708+0000","last_fullsized":"2024-09-16T10:34:27.200708+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:15.333230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117618+0000","last_change":"2024-09-16T10:33:19.871992+0000","last_active":"2024-09-16T10:33:24.117618+0000","last_peered":"2024-09-16T10:33:24.117618+0000","last_clean":"2024-09-16T10:33:24.117618+0000","last_became_active":"2024-09-16T10:33:19.871755+0000","last_became_peered":"2024-09-16T10:33:19.871755+0000","last_unstale":"2024-09-16T10:33:24.117618+0000","last_undegraded":"2024-09-16T10:33:24.117618+0000","last_fullsized":"2024-09-16T10:33:24.117618+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:18.251978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405269+0000","last_change":"2024-09-16T10:33:15.848251+0000","last_active":"2024-09-16T10:33:24.405269+0000","last_peered":"2024-09-16T10:33:24.405269+0000","last_clean":"2024-09-16T10:33:24.405269+0000","last_became_active":"2024-09-16T10:33:15.847981+0000","last_became_peered":"2024-09-16T10:33:15.847981+0000","last_unstale":"2024-09-16T10:33:24.405269+0000","last_undegraded":"2024-09-16T10:33:24.405269+0000","last_fullsized":"2024-09-16T10:33:24.405269+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:44.218431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403636+0000","last_change":"2024-09-16T10:33:13.830092+0000","last_active":"2024-09-16T10:33:24.403636+0000","last_peered":"2024-09-16T10:33:24.403636+0000","last_clean":"2024-09-16T10:33:24.403636+0000","last_became_active":"2024-09-16T10:33:13.829790+0000","last_became_peered":"2024-09-16T10:33:13.829790+0000","last_unstale":"2024-09-16T10:33:24.403636+0000","last_undegraded":"2024-09-16T10:33:24.403636+0000","last_fullsized":"2024-09-16T10:33:24.403636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:39.118186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117804+0000","last_change":"2024-09-16T10:33:17.862965+0000","last_active":"2024-09-16T10:33:24.117804+0000","last_peered":"2024-09-16T10:33:24.117804+0000","last_clean":"2024-09-16T10:33:24.117804+0000","last_became_active":"2024-09-16T10:33:17.862684+0000","last_became_peered":"2024-09-16T10:33:17.862684+0000","last_unstale":"2024-09-16T10:33:24.117804+0000","last_undegraded":"2024-09-16T10:33:24.117804+0000","last_fullsized":"2024-09-16T10:33:24.117804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:27.999100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122000+0000","last_change":"2024-09-16T10:33:19.908083+0000","last_active":"2024-09-16T10:33:24.122000+0000","last_peered":"2024-09-16T10:33:24.122000+0000","last_clean":"2024-09-16T10:33:24.122000+0000","last_became_active":"2024-09-16T10:33:19.907721+0000","last_became_peered":"2024-09-16T10:33:19.907721+0000","last_unstale":"2024-09-16T10:33:24.122000+0000","last_undegraded":"2024-09-16T10:33:24.122000+0000","last_fullsized":"2024-09-16T10:33:24.122000+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:06:18.374644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":38,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404409+0000","last_change":"2024-09-16T10:33:15.847080+0000","last_active":"2024-09-16T10:33:24.404409+0000","last_peered":"2024-09-16T10:33:24.404409+0000","last_clean":"2024-09-16T10:33:24.404409+0000","last_became_active":"2024-09-16T10:33:15.846198+0000","last_became_peered":"2024-09-16T10:33:15.846198+0000","last_unstale":"2024-09-16T10:33:24.404409+0000","last_undegraded":"2024-09-16T10:33:24.404409+0000","last_fullsized":"2024-09-16T10:33:24.404409+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:31.743731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404227+0000","last_change":"2024-09-16T10:33:13.823815+0000","last_active":"2024-09-16T10:33:24.404227+0000","last_peered":"2024-09-16T10:33:24.404227+0000","last_clean":"2024-09-16T10:33:24.404227+0000","last_became_active":"2024-09-16T10:33:13.823321+0000","last_became_peered":"2024-09-16T10:33:13.823321+0000","last_unstale":"2024-09-16T10:33:24.404227+0000","last_undegraded":"2024-09-16T10:33:24.404227+0000","last_fullsized":"2024-09-16T10:33:24.404227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:22.783191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200611+0000","last_change":"2024-09-16T10:33:17.858121+0000","last_active":"2024-09-16T10:34:27.200611+0000","last_peered":"2024-09-16T10:34:27.200611+0000","last_clean":"2024-09-16T10:34:27.200611+0000","last_became_active":"2024-09-16T10:33:17.857853+0000","last_became_peered":"2024-09-16T10:33:17.857853+0000","last_unstale":"2024-09-16T10:34:27.200611+0000","last_undegraded":"2024-09-16T10:34:27.200611+0000","last_fullsized":"2024-09-16T10:34:27.200611+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:59.348644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404658+0000","last_change":"2024-09-16T10:33:19.909477+0000","last_active":"2024-09-16T10:33:24.404658+0000","last_peered":"2024-09-16T10:33:24.404658+0000","last_clean":"2024-09-16T10:33:24.404658+0000","last_became_active":"2024-09-16T10:33:19.909194+0000","last_became_peered":"2024-09-16T10:33:19.909194+0000","last_unstale":"2024-09-16T10:33:24.404658+0000","last_undegraded":"2024-09-16T10:33:24.404658+0000","last_fullsized":"2024-09-16T10:33:24.404658+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:13.158503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":49,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121523+0000","last_change":"2024-09-16T10:33:15.850983+0000","last_active":"2024-09-16T10:33:24.121523+0000","last_peered":"2024-09-16T10:33:24.121523+0000","last_clean":"2024-09-16T10:33:24.121523+0000","last_became_active":"2024-09-16T10:33:15.850720+0000","last_became_peered":"2024-09-16T10:33:15.850720+0000","last_unstale":"2024-09-16T10:33:24.121523+0000","last_undegraded":"2024-09-16T10:33:24.121523+0000","last_fullsized":"2024-09-16T10:33:24.121523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:37:57.092520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122722+0000","last_change":"2024-09-16T10:33:13.821362+0000","last_active":"2024-09-16T10:33:24.122722+0000","last_peered":"2024-09-16T10:33:24.122722+0000","last_clean":"2024-09-16T10:33:24.122722+0000","last_became_active":"2024-09-16T10:33:13.821102+0000","last_became_peered":"2024-09-16T10:33:13.821102+0000","last_unstale":"2024-09-16T10:33:24.122722+0000","last_undegraded":"2024-09-16T10:33:24.122722+0000","last_fullsized":"2024-09-16T10:33:24.122722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:41:38.478745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404320+0000","last_change":"2024-09-16T10:33:17.865528+0000","last_active":"2024-09-16T10:33:24.404320+0000","last_peered":"2024-09-16T10:33:24.404320+0000","last_clean":"2024-09-16T10:33:24.404320+0000","last_became_active":"2024-09-16T10:33:17.865249+0000","last_became_peered":"2024-09-16T10:33:17.865249+0000","last_unstale":"2024-09-16T10:33:24.404320+0000","last_undegraded":"2024-09-16T10:33:24.404320+0000","last_fullsized":"2024-09-16T10:33:24.404320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:08:27.575571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403753+0000","last_change":"2024-09-16T10:33:19.872124+0000","last_active":"2024-09-16T10:33:24.403753+0000","last_peered":"2024-09-16T10:33:24.403753+0000","last_clean":"2024-09-16T10:33:24.403753+0000","last_became_active":"2024-09-16T10:33:19.871812+0000","last_became_peered":"2024-09-16T10:33:19.871812+0000","last_unstale":"2024-09-16T10:33:24.403753+0000","last_undegraded":"2024-09-16T10:33:24.403753+0000","last_fullsized":"2024-09-16T10:33:24.403753+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:54:05.585783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":53,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404700+0000","last_change":"2024-09-16T10:33:15.846613+0000","last_active":"2024-09-16T10:33:24.404700+0000","last_peered":"2024-09-16T10:33:24.404700+0000","last_clean":"2024-09-16T10:33:24.404700+0000","last_became_active":"2024-09-16T10:33:15.846321+0000","last_became_peered":"2024-09-16T10:33:15.846321+0000","last_unstale":"2024-09-16T10:33:24.404700+0000","last_undegraded":"2024-09-16T10:33:24.404700+0000","last_fullsized":"2024-09-16T10:33:24.404700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:38.391756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405358+0000","last_change":"2024-09-16T10:33:13.830471+0000","last_active":"2024-09-16T10:33:24.405358+0000","last_peered":"2024-09-16T10:33:24.405358+0000","last_clean":"2024-09-16T10:33:24.405358+0000","last_became_active":"2024-09-16T10:33:13.829598+0000","last_became_peered":"2024-09-16T10:33:13.829598+0000","last_unstale":"2024-09-16T10:33:24.405358+0000","last_undegraded":"2024-09-16T10:33:24.405358+0000","last_fullsized":"2024-09-16T10:33:24.405358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:55.836948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405167+0000","last_change":"2024-09-16T10:33:17.858857+0000","last_active":"2024-09-16T10:33:24.405167+0000","last_peered":"2024-09-16T10:33:24.405167+0000","last_clean":"2024-09-16T10:33:24.405167+0000","last_became_active":"2024-09-16T10:33:17.858598+0000","last_became_peered":"2024-09-16T10:33:17.858598+0000","last_unstale":"2024-09-16T10:33:24.405167+0000","last_undegraded":"2024-09-16T10:33:24.405167+0000","last_fullsized":"2024-09-16T10:33:24.405167+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:48.093085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120387+0000","last_change":"2024-09-16T10:33:19.907978+0000","last_active":"2024-09-16T10:33:24.120387+0000","last_peered":"2024-09-16T10:33:24.120387+0000","last_clean":"2024-09-16T10:33:24.120387+0000","last_became_active":"2024-09-16T10:33:19.907676+0000","last_became_peered":"2024-09-16T10:33:19.907676+0000","last_unstale":"2024-09-16T10:33:24.120387+0000","last_undegraded":"2024-09-16T10:33:24.120387+0000","last_fullsized":"2024-09-16T10:33:24.120387+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:45.261026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120992+0000","last_change":"2024-09-16T10:33:15.845291+0000","last_active":"2024-09-16T10:33:24.120992+0000","last_peered":"2024-09-16T10:33:24.120992+0000","last_clean":"2024-09-16T10:33:24.120992+0000","last_became_active":"2024-09-16T10:33:15.845011+0000","last_became_peered":"2024-09-16T10:33:15.845011+0000","last_unstale":"2024-09-16T10:33:24.120992+0000","last_undegraded":"2024-09-16T10:33:24.120992+0000","last_fullsized":"2024-09-16T10:33:24.120992+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:31.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121261+0000","last_change":"2024-09-16T10:33:13.827724+0000","last_active":"2024-09-16T10:33:24.121261+0000","last_peered":"2024-09-16T10:33:24.121261+0000","last_clean":"2024-09-16T10:33:24.121261+0000","last_became_active":"2024-09-16T10:33:13.827401+0000","last_became_peered":"2024-09-16T10:33:13.827401+0000","last_unstale":"2024-09-16T10:33:24.121261+0000","last_undegraded":"2024-09-16T10:33:24.121261+0000","last_fullsized":"2024-09-16T10:33:24.121261+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:59.906669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200591+0000","last_change":"2024-09-16T10:33:17.859954+0000","last_active":"2024-09-16T10:34:27.200591+0000","last_peered":"2024-09-16T10:34:27.200591+0000","last_clean":"2024-09-16T10:34:27.200591+0000","last_became_active":"2024-09-16T10:33:17.859634+0000","last_became_peered":"2024-09-16T10:33:17.859634+0000","last_unstale":"2024-09-16T10:34:27.200591+0000","last_undegraded":"2024-09-16T10:34:27.200591+0000","last_fullsized":"2024-09-16T10:34:27.200591+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:26.862187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403725+0000","last_change":"2024-09-16T10:33:19.877161+0000","last_active":"2024-09-16T10:33:24.403725+0000","last_peered":"2024-09-16T10:33:24.403725+0000","last_clean":"2024-09-16T10:33:24.403725+0000","last_became_active":"2024-09-16T10:33:19.876609+0000","last_became_peered":"2024-09-16T10:33:19.876609+0000","last_unstale":"2024-09-16T10:33:24.403725+0000","last_undegraded":"2024-09-16T10:33:24.403725+0000","last_fullsized":"2024-09-16T10:33:24.403725+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:08:43.855631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":81,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200607+0000","last_change":"2024-09-16T10:33:15.850954+0000","last_active":"2024-09-16T10:34:27.200607+0000","last_peered":"2024-09-16T10:34:27.200607+0000","last_clean":"2024-09-16T10:34:27.200607+0000","last_became_active":"2024-09-16T10:33:15.850147+0000","last_became_peered":"2024-09-16T10:33:15.850147+0000","last_unstale":"2024-09-16T10:34:27.200607+0000","last_undegraded":"2024-09-16T10:34:27.200607+0000","last_fullsized":"2024-09-16T10:34:27.200607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:41.061633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117540+0000","last_change":"2024-09-16T10:33:13.826857+0000","last_active":"2024-09-16T10:33:24.117540+0000","last_peered":"2024-09-16T10:33:24.117540+0000","last_clean":"2024-09-16T10:33:24.117540+0000","last_became_active":"2024-09-16T10:33:13.826596+0000","last_became_peered":"2024-09-16T10:33:13.826596+0000","last_unstale":"2024-09-16T10:33:24.117540+0000","last_undegraded":"2024-09-16T10:33:24.117540+0000","last_fullsized":"2024-09-16T10:33:24.117540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:52.557415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404482+0000","last_change":"2024-09-16T10:33:17.863382+0000","last_active":"2024-09-16T10:33:24.404482+0000","last_peered":"2024-09-16T10:33:24.404482+0000","last_clean":"2024-09-16T10:33:24.404482+0000","last_became_active":"2024-09-16T10:33:17.863013+0000","last_became_peered":"2024-09-16T10:33:17.863013+0000","last_unstale":"2024-09-16T10:33:24.404482+0000","last_undegraded":"2024-09-16T10:33:24.404482+0000","last_fullsized":"2024-09-16T10:33:24.404482+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:29:07.651347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122091+0000","last_change":"2024-09-16T10:33:19.872874+0000","last_active":"2024-09-16T10:33:24.122091+0000","last_peered":"2024-09-16T10:33:24.122091+0000","last_clean":"2024-09-16T10:33:24.122091+0000","last_became_active":"2024-09-16T10:33:19.872557+0000","last_became_peered":"2024-09-16T10:33:19.872557+0000","last_unstale":"2024-09-16T10:33:24.122091+0000","last_undegraded":"2024-09-16T10:33:24.122091+0000","last_fullsized":"2024-09-16T10:33:24.122091+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:16.485585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":44,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122158+0000","last_change":"2024-09-16T10:33:15.849555+0000","last_active":"2024-09-16T10:33:24.122158+0000","last_peered":"2024-09-16T10:33:24.122158+0000","last_clean":"2024-09-16T10:33:24.122158+0000","last_became_active":"2024-09-16T10:33:15.848762+0000","last_became_peered":"2024-09-16T10:33:15.848762+0000","last_unstale":"2024-09-16T10:33:24.122158+0000","last_undegraded":"2024-09-16T10:33:24.122158+0000","last_fullsized":"2024-09-16T10:33:24.122158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:47:06.625488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"69'5","reported_seq":101,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:33.388605+0000","last_change":"2024-09-16T10:33:17.848105+0000","last_active":"2024-09-16T10:34:33.388605+0000","last_peered":"2024-09-16T10:34:33.388605+0000","last_clean":"2024-09-16T10:34:33.388605+0000","last_became_active":"2024-09-16T10:33:13.819415+0000","last_became_peered":"2024-09-16T10:33:13.819415+0000","last_unstale":"2024-09-16T10:34:33.388605+0000","last_undegraded":"2024-09-16T10:34:33.388605+0000","last_fullsized":"2024-09-16T10:34:33.388605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:54.771452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002177357,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404364+0000","last_change":"2024-09-16T10:33:13.830338+0000","last_active":"2024-09-16T10:33:24.404364+0000","last_peered":"2024-09-16T10:33:24.404364+0000","last_clean":"2024-09-16T10:33:24.404364+0000","last_became_active":"2024-09-16T10:33:13.830063+0000","last_became_peered":"2024-09-16T10:33:13.830063+0000","last_unstale":"2024-09-16T10:33:24.404364+0000","last_undegraded":"2024-09-16T10:33:24.404364+0000","last_fullsized":"2024-09-16T10:33:24.404364+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:17:38.454962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404921+0000","last_change":"2024-09-16T10:33:17.856564+0000","last_active":"2024-09-16T10:33:24.404921+0000","last_peered":"2024-09-16T10:33:24.404921+0000","last_clean":"2024-09-16T10:33:24.404921+0000","last_became_active":"2024-09-16T10:33:17.856131+0000","last_became_peered":"2024-09-16T10:33:17.856131+0000","last_unstale":"2024-09-16T10:33:24.404921+0000","last_undegraded":"2024-09-16T10:33:24.404921+0000","last_fullsized":"2024-09-16T10:33:24.404921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:19.224434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405056+0000","last_change":"2024-09-16T10:33:19.907992+0000","last_active":"2024-09-16T10:33:24.405056+0000","last_peered":"2024-09-16T10:33:24.405056+0000","last_clean":"2024-09-16T10:33:24.405056+0000","last_became_active":"2024-09-16T10:33:19.907706+0000","last_became_peered":"2024-09-16T10:33:19.907706+0000","last_unstale":"2024-09-16T10:33:24.405056+0000","last_undegraded":"2024-09-16T10:33:24.405056+0000","last_fullsized":"2024-09-16T10:33:24.405056+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:51:51.114352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":35,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405386+0000","last_change":"2024-09-16T10:33:15.846739+0000","last_active":"2024-09-16T10:33:24.405386+0000","last_peered":"2024-09-16T10:33:24.405386+0000","last_clean":"2024-09-16T10:33:24.405386+0000","last_became_active":"2024-09-16T10:33:15.846017+0000","last_became_peered":"2024-09-16T10:33:15.846017+0000","last_unstale":"2024-09-16T10:33:24.405386+0000","last_undegraded":"2024-09-16T10:33:24.405386+0000","last_fullsized":"2024-09-16T10:33:24.405386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:33.275546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'1","reported_seq":25,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404363+0000","last_change":"2024-09-16T10:33:17.852278+0000","last_active":"2024-09-16T10:33:24.404363+0000","last_peered":"2024-09-16T10:33:24.404363+0000","last_clean":"2024-09-16T10:33:24.404363+0000","last_became_active":"2024-09-16T10:33:13.833909+0000","last_became_peered":"2024-09-16T10:33:13.833909+0000","last_unstale":"2024-09-16T10:33:24.404363+0000","last_undegraded":"2024-09-16T10:33:24.404363+0000","last_fullsized":"2024-09-16T10:33:24.404363+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:08.748999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037669100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'1","reported_seq":24,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403712+0000","last_change":"2024-09-16T10:33:13.829887+0000","last_active":"2024-09-16T10:33:24.403712+0000","last_peered":"2024-09-16T10:33:24.403712+0000","last_clean":"2024-09-16T10:33:24.403712+0000","last_became_active":"2024-09-16T10:33:13.829591+0000","last_became_peered":"2024-09-16T10:33:13.829591+0000","last_unstale":"2024-09-16T10:33:24.403712+0000","last_undegraded":"2024-09-16T10:33:24.403712+0000","last_fullsized":"2024-09-16T10:33:24.403712+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:28.484307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117828+0000","last_change":"2024-09-16T10:33:17.860155+0000","last_active":"2024-09-16T10:33:24.117828+0000","last_peered":"2024-09-16T10:33:24.117828+0000","last_clean":"2024-09-16T10:33:24.117828+0000","last_became_active":"2024-09-16T10:33:17.859769+0000","last_became_peered":"2024-09-16T10:33:17.859769+0000","last_unstale":"2024-09-16T10:33:24.117828+0000","last_undegraded":"2024-09-16T10:33:24.117828+0000","last_fullsized":"2024-09-16T10:33:24.117828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:30.480504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"68'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120990+0000","last_change":"2024-09-16T10:33:19.873795+0000","last_active":"2024-09-16T10:33:24.120990+0000","last_peered":"2024-09-16T10:33:24.120990+0000","last_clean":"2024-09-16T10:33:24.120990+0000","last_became_active":"2024-09-16T10:33:19.873433+0000","last_became_peered":"2024-09-16T10:33:19.873433+0000","last_unstale":"2024-09-16T10:33:24.120990+0000","last_undegraded":"2024-09-16T10:33:24.120990+0000","last_fullsized":"2024-09-16T10:33:24.120990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:20:54.213371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"68'14","reported_seq":39,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404285+0000","last_change":"2024-09-16T10:33:15.844147+0000","last_active":"2024-09-16T10:33:24.404285+0000","last_peered":"2024-09-16T10:33:24.404285+0000","last_clean":"2024-09-16T10:33:24.404285+0000","last_became_active":"2024-09-16T10:33:15.843754+0000","last_became_peered":"2024-09-16T10:33:15.843754+0000","last_unstale":"2024-09-16T10:33:24.404285+0000","last_undegraded":"2024-09-16T10:33:24.404285+0000","last_fullsized":"2024-09-16T10:33:24.404285+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:17.719923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122103+0000","last_change":"2024-09-16T10:33:17.857138+0000","last_active":"2024-09-16T10:33:24.122103+0000","last_peered":"2024-09-16T10:33:24.122103+0000","last_clean":"2024-09-16T10:33:24.122103+0000","last_became_active":"2024-09-16T10:33:13.829882+0000","last_became_peered":"2024-09-16T10:33:13.829882+0000","last_unstale":"2024-09-16T10:33:24.122103+0000","last_undegraded":"2024-09-16T10:33:24.122103+0000","last_fullsized":"2024-09-16T10:33:24.122103+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:48:49.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018861469999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117501+0000","last_change":"2024-09-16T10:33:13.832221+0000","last_active":"2024-09-16T10:33:24.117501+0000","last_peered":"2024-09-16T10:33:24.117501+0000","last_clean":"2024-09-16T10:33:24.117501+0000","last_became_active":"2024-09-16T10:33:13.831955+0000","last_became_peered":"2024-09-16T10:33:13.831955+0000","last_unstale":"2024-09-16T10:33:24.117501+0000","last_undegraded":"2024-09-16T10:33:24.117501+0000","last_fullsized":"2024-09-16T10:33:24.117501+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:07.925458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'188","reported_seq":334,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:33:51.058979+0000","last_change":"2024-09-16T10:33:02.009627+0000","last_active":"2024-09-16T10:33:51.058979+0000","last_peered":"2024-09-16T10:33:51.058979+0000","last_clean":"2024-09-16T10:33:51.058979+0000","last_became_active":"2024-09-16T10:33:02.009112+0000","last_became_peered":"2024-09-16T10:33:02.009112+0000","last_unstale":"2024-09-16T10:33:51.058979+0000","last_undegraded":"2024-09-16T10:33:51.058979+0000","last_fullsized":"2024-09-16T10:33:51.058979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:31:00.046657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:31:00.046657+0000","last_clean_scrub_stamp":"2024-09-16T10:31:00.046657+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:51.264238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405068+0000","last_change":"2024-09-16T10:33:17.856219+0000","last_active":"2024-09-16T10:33:24.405068+0000","last_peered":"2024-09-16T10:33:24.405068+0000","last_clean":"2024-09-16T10:33:24.405068+0000","last_became_active":"2024-09-16T10:33:17.855883+0000","last_became_peered":"2024-09-16T10:33:17.855883+0000","last_unstale":"2024-09-16T10:33:24.405068+0000","last_undegraded":"2024-09-16T10:33:24.405068+0000","last_fullsized":"2024-09-16T10:33:24.405068+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:04:09.771912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117876+0000","last_change":"2024-09-16T10:33:19.870198+0000","last_active":"2024-09-16T10:33:24.117876+0000","last_peered":"2024-09-16T10:33:24.117876+0000","last_clean":"2024-09-16T10:33:24.117876+0000","last_became_active":"2024-09-16T10:33:19.869936+0000","last_became_peered":"2024-09-16T10:33:19.869936+0000","last_unstale":"2024-09-16T10:33:24.117876+0000","last_undegraded":"2024-09-16T10:33:24.117876+0000","last_fullsized":"2024-09-16T10:33:24.117876+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:26.776086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":48,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405339+0000","last_change":"2024-09-16T10:33:15.845378+0000","last_active":"2024-09-16T10:33:24.405339+0000","last_peered":"2024-09-16T10:33:24.405339+0000","last_clean":"2024-09-16T10:33:24.405339+0000","last_became_active":"2024-09-16T10:33:15.844800+0000","last_became_peered":"2024-09-16T10:33:15.844800+0000","last_unstale":"2024-09-16T10:33:24.405339+0000","last_undegraded":"2024-09-16T10:33:24.405339+0000","last_fullsized":"2024-09-16T10:33:24.405339+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:05.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404921+0000","last_change":"2024-09-16T10:33:13.827575+0000","last_active":"2024-09-16T10:33:24.404921+0000","last_peered":"2024-09-16T10:33:24.404921+0000","last_clean":"2024-09-16T10:33:24.404921+0000","last_became_active":"2024-09-16T10:33:13.827277+0000","last_became_peered":"2024-09-16T10:33:13.827277+0000","last_unstale":"2024-09-16T10:33:24.404921+0000","last_undegraded":"2024-09-16T10:33:24.404921+0000","last_fullsized":"2024-09-16T10:33:24.404921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:34:59.152384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404298+0000","last_change":"2024-09-16T10:33:17.858189+0000","last_active":"2024-09-16T10:33:24.404298+0000","last_peered":"2024-09-16T10:33:24.404298+0000","last_clean":"2024-09-16T10:33:24.404298+0000","last_became_active":"2024-09-16T10:33:17.857563+0000","last_became_peered":"2024-09-16T10:33:17.857563+0000","last_unstale":"2024-09-16T10:33:24.404298+0000","last_undegraded":"2024-09-16T10:33:24.404298+0000","last_fullsized":"2024-09-16T10:33:24.404298+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:31.954377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403799+0000","last_change":"2024-09-16T10:33:19.872038+0000","last_active":"2024-09-16T10:33:24.403799+0000","last_peered":"2024-09-16T10:33:24.403799+0000","last_clean":"2024-09-16T10:33:24.403799+0000","last_became_active":"2024-09-16T10:33:19.871772+0000","last_became_peered":"2024-09-16T10:33:19.871772+0000","last_unstale":"2024-09-16T10:33:24.403799+0000","last_undegraded":"2024-09-16T10:33:24.403799+0000","last_fullsized":"2024-09-16T10:33:24.403799+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:39.278027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404591+0000","last_change":"2024-09-16T10:33:15.848259+0000","last_active":"2024-09-16T10:33:24.404591+0000","last_peered":"2024-09-16T10:33:24.404591+0000","last_clean":"2024-09-16T10:33:24.404591+0000","last_became_active":"2024-09-16T10:33:15.847953+0000","last_became_peered":"2024-09-16T10:33:15.847953+0000","last_unstale":"2024-09-16T10:33:24.404591+0000","last_undegraded":"2024-09-16T10:33:24.404591+0000","last_fullsized":"2024-09-16T10:33:24.404591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:39:39.168243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122005+0000","last_change":"2024-09-16T10:33:13.824507+0000","last_active":"2024-09-16T10:33:24.122005+0000","last_peered":"2024-09-16T10:33:24.122005+0000","last_clean":"2024-09-16T10:33:24.122005+0000","last_became_active":"2024-09-16T10:33:13.824098+0000","last_became_peered":"2024-09-16T10:33:13.824098+0000","last_unstale":"2024-09-16T10:33:24.122005+0000","last_undegraded":"2024-09-16T10:33:24.122005+0000","last_fullsized":"2024-09-16T10:33:24.122005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:53.987321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404118+0000","last_change":"2024-09-16T10:33:17.861312+0000","last_active":"2024-09-16T10:33:24.404118+0000","last_peered":"2024-09-16T10:33:24.404118+0000","last_clean":"2024-09-16T10:33:24.404118+0000","last_became_active":"2024-09-16T10:33:17.861006+0000","last_became_peered":"2024-09-16T10:33:17.861006+0000","last_unstale":"2024-09-16T10:33:24.404118+0000","last_undegraded":"2024-09-16T10:33:24.404118+0000","last_fullsized":"2024-09-16T10:33:24.404118+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:11:22.858849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404202+0000","last_change":"2024-09-16T10:33:19.876459+0000","last_active":"2024-09-16T10:33:24.404202+0000","last_peered":"2024-09-16T10:33:24.404202+0000","last_clean":"2024-09-16T10:33:24.404202+0000","last_became_active":"2024-09-16T10:33:19.876171+0000","last_became_peered":"2024-09-16T10:33:19.876171+0000","last_unstale":"2024-09-16T10:33:24.404202+0000","last_undegraded":"2024-09-16T10:33:24.404202+0000","last_fullsized":"2024-09-16T10:33:24.404202+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:47.206200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117195+0000","last_change":"2024-09-16T10:33:15.846052+0000","last_active":"2024-09-16T10:33:24.117195+0000","last_peered":"2024-09-16T10:33:24.117195+0000","last_clean":"2024-09-16T10:33:24.117195+0000","last_became_active":"2024-09-16T10:33:15.845756+0000","last_became_peered":"2024-09-16T10:33:15.845756+0000","last_unstale":"2024-09-16T10:33:24.117195+0000","last_undegraded":"2024-09-16T10:33:24.117195+0000","last_fullsized":"2024-09-16T10:33:24.117195+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:35.349667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"61'1","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405044+0000","last_change":"2024-09-16T10:33:13.823475+0000","last_active":"2024-09-16T10:33:24.405044+0000","last_peered":"2024-09-16T10:33:24.405044+0000","last_clean":"2024-09-16T10:33:24.405044+0000","last_became_active":"2024-09-16T10:33:13.823203+0000","last_became_peered":"2024-09-16T10:33:13.823203+0000","last_unstale":"2024-09-16T10:33:24.405044+0000","last_undegraded":"2024-09-16T10:33:24.405044+0000","last_fullsized":"2024-09-16T10:33:24.405044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:40.846895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200728+0000","last_change":"2024-09-16T10:33:17.859546+0000","last_active":"2024-09-16T10:34:27.200728+0000","last_peered":"2024-09-16T10:34:27.200728+0000","last_clean":"2024-09-16T10:34:27.200728+0000","last_became_active":"2024-09-16T10:33:17.859246+0000","last_became_peered":"2024-09-16T10:33:17.859246+0000","last_unstale":"2024-09-16T10:34:27.200728+0000","last_undegraded":"2024-09-16T10:34:27.200728+0000","last_fullsized":"2024-09-16T10:34:27.200728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:59.378380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117270+0000","last_change":"2024-09-16T10:33:19.908097+0000","last_active":"2024-09-16T10:33:24.117270+0000","last_peered":"2024-09-16T10:33:24.117270+0000","last_clean":"2024-09-16T10:33:24.117270+0000","last_became_active":"2024-09-16T10:33:19.907747+0000","last_became_peered":"2024-09-16T10:33:19.907747+0000","last_unstale":"2024-09-16T10:33:24.117270+0000","last_undegraded":"2024-09-16T10:33:24.117270+0000","last_fullsized":"2024-09-16T10:33:24.117270+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:29.122676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":40,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404318+0000","last_change":"2024-09-16T10:33:15.854009+0000","last_active":"2024-09-16T10:33:24.404318+0000","last_peered":"2024-09-16T10:33:24.404318+0000","last_clean":"2024-09-16T10:33:24.404318+0000","last_became_active":"2024-09-16T10:33:15.853645+0000","last_became_peered":"2024-09-16T10:33:15.853645+0000","last_unstale":"2024-09-16T10:33:24.404318+0000","last_undegraded":"2024-09-16T10:33:24.404318+0000","last_fullsized":"2024-09-16T10:33:24.404318+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:03.443239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404877+0000","last_change":"2024-09-16T10:33:13.828001+0000","last_active":"2024-09-16T10:33:24.404877+0000","last_peered":"2024-09-16T10:33:24.404877+0000","last_clean":"2024-09-16T10:33:24.404877+0000","last_became_active":"2024-09-16T10:33:13.827735+0000","last_became_peered":"2024-09-16T10:33:13.827735+0000","last_unstale":"2024-09-16T10:33:24.404877+0000","last_undegraded":"2024-09-16T10:33:24.404877+0000","last_fullsized":"2024-09-16T10:33:24.404877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:10.056999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403827+0000","last_change":"2024-09-16T10:33:17.854722+0000","last_active":"2024-09-16T10:33:24.403827+0000","last_peered":"2024-09-16T10:33:24.403827+0000","last_clean":"2024-09-16T10:33:24.403827+0000","last_became_active":"2024-09-16T10:33:17.854452+0000","last_became_peered":"2024-09-16T10:33:17.854452+0000","last_unstale":"2024-09-16T10:33:24.403827+0000","last_undegraded":"2024-09-16T10:33:24.403827+0000","last_fullsized":"2024-09-16T10:33:24.403827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:38:29.759276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122490+0000","last_change":"2024-09-16T10:33:19.875323+0000","last_active":"2024-09-16T10:33:24.122490+0000","last_peered":"2024-09-16T10:33:24.122490+0000","last_clean":"2024-09-16T10:33:24.122490+0000","last_became_active":"2024-09-16T10:33:19.874868+0000","last_became_peered":"2024-09-16T10:33:19.874868+0000","last_unstale":"2024-09-16T10:33:24.122490+0000","last_undegraded":"2024-09-16T10:33:24.122490+0000","last_fullsized":"2024-09-16T10:33:24.122490+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:07.941852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405359+0000","last_change":"2024-09-16T10:33:15.854049+0000","last_active":"2024-09-16T10:33:24.405359+0000","last_peered":"2024-09-16T10:33:24.405359+0000","last_clean":"2024-09-16T10:33:24.405359+0000","last_became_active":"2024-09-16T10:33:15.853789+0000","last_became_peered":"2024-09-16T10:33:15.853789+0000","last_unstale":"2024-09-16T10:33:24.405359+0000","last_undegraded":"2024-09-16T10:33:24.405359+0000","last_fullsized":"2024-09-16T10:33:24.405359+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:04.753452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403872+0000","last_change":"2024-09-16T10:33:13.827742+0000","last_active":"2024-09-16T10:33:24.403872+0000","last_peered":"2024-09-16T10:33:24.403872+0000","last_clean":"2024-09-16T10:33:24.403872+0000","last_became_active":"2024-09-16T10:33:13.827292+0000","last_became_peered":"2024-09-16T10:33:13.827292+0000","last_unstale":"2024-09-16T10:33:24.403872+0000","last_undegraded":"2024-09-16T10:33:24.403872+0000","last_fullsized":"2024-09-16T10:33:24.403872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:23.812314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121732+0000","last_change":"2024-09-16T10:33:17.857595+0000","last_active":"2024-09-16T10:33:24.121732+0000","last_peered":"2024-09-16T10:33:24.121732+0000","last_clean":"2024-09-16T10:33:24.121732+0000","last_became_active":"2024-09-16T10:33:17.857301+0000","last_became_peered":"2024-09-16T10:33:17.857301+0000","last_unstale":"2024-09-16T10:33:24.121732+0000","last_undegraded":"2024-09-16T10:33:24.121732+0000","last_fullsized":"2024-09-16T10:33:24.121732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:14.989023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403995+0000","last_change":"2024-09-16T10:33:19.869229+0000","last_active":"2024-09-16T10:33:24.403995+0000","last_peered":"2024-09-16T10:33:24.403995+0000","last_clean":"2024-09-16T10:33:24.403995+0000","last_became_active":"2024-09-16T10:33:19.868946+0000","last_became_peered":"2024-09-16T10:33:19.868946+0000","last_unstale":"2024-09-16T10:33:24.403995+0000","last_undegraded":"2024-09-16T10:33:24.403995+0000","last_fullsized":"2024-09-16T10:33:24.403995+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:10.003520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404900+0000","last_change":"2024-09-16T10:33:13.825980+0000","last_active":"2024-09-16T10:33:24.404900+0000","last_peered":"2024-09-16T10:33:24.404900+0000","last_clean":"2024-09-16T10:33:24.404900+0000","last_became_active":"2024-09-16T10:33:13.825733+0000","last_became_peered":"2024-09-16T10:33:13.825733+0000","last_unstale":"2024-09-16T10:33:24.404900+0000","last_undegraded":"2024-09-16T10:33:24.404900+0000","last_fullsized":"2024-09-16T10:33:24.404900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:49.775857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":26,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121905+0000","last_change":"2024-09-16T10:33:15.848828+0000","last_active":"2024-09-16T10:33:24.121905+0000","last_peered":"2024-09-16T10:33:24.121905+0000","last_clean":"2024-09-16T10:33:24.121905+0000","last_became_active":"2024-09-16T10:33:15.848528+0000","last_became_peered":"2024-09-16T10:33:15.848528+0000","last_unstale":"2024-09-16T10:33:24.121905+0000","last_undegraded":"2024-09-16T10:33:24.121905+0000","last_fullsized":"2024-09-16T10:33:24.121905+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:14.855054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117754+0000","last_change":"2024-09-16T10:33:17.861296+0000","last_active":"2024-09-16T10:33:24.117754+0000","last_peered":"2024-09-16T10:33:24.117754+0000","last_clean":"2024-09-16T10:33:24.117754+0000","last_became_active":"2024-09-16T10:33:17.861029+0000","last_became_peered":"2024-09-16T10:33:17.861029+0000","last_unstale":"2024-09-16T10:33:24.117754+0000","last_undegraded":"2024-09-16T10:33:24.117754+0000","last_fullsized":"2024-09-16T10:33:24.117754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:41:09.753387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404985+0000","last_change":"2024-09-16T10:33:19.909519+0000","last_active":"2024-09-16T10:33:24.404985+0000","last_peered":"2024-09-16T10:33:24.404985+0000","last_clean":"2024-09-16T10:33:24.404985+0000","last_became_active":"2024-09-16T10:33:19.909186+0000","last_became_peered":"2024-09-16T10:33:19.909186+0000","last_unstale":"2024-09-16T10:33:24.404985+0000","last_undegraded":"2024-09-16T10:33:24.404985+0000","last_fullsized":"2024-09-16T10:33:24.404985+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:22.102551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122407+0000","last_change":"2024-09-16T10:33:15.848821+0000","last_active":"2024-09-16T10:33:24.122407+0000","last_peered":"2024-09-16T10:33:24.122407+0000","last_clean":"2024-09-16T10:33:24.122407+0000","last_became_active":"2024-09-16T10:33:15.848521+0000","last_became_peered":"2024-09-16T10:33:15.848521+0000","last_unstale":"2024-09-16T10:33:24.122407+0000","last_undegraded":"2024-09-16T10:33:24.122407+0000","last_fullsized":"2024-09-16T10:33:24.122407+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:16.660941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117537+0000","last_change":"2024-09-16T10:33:13.831413+0000","last_active":"2024-09-16T10:33:24.117537+0000","last_peered":"2024-09-16T10:33:24.117537+0000","last_clean":"2024-09-16T10:33:24.117537+0000","last_became_active":"2024-09-16T10:33:13.831158+0000","last_became_peered":"2024-09-16T10:33:13.831158+0000","last_unstale":"2024-09-16T10:33:24.117537+0000","last_undegraded":"2024-09-16T10:33:24.117537+0000","last_fullsized":"2024-09-16T10:33:24.117537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:02.031293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200772+0000","last_change":"2024-09-16T10:33:17.862043+0000","last_active":"2024-09-16T10:34:27.200772+0000","last_peered":"2024-09-16T10:34:27.200772+0000","last_clean":"2024-09-16T10:34:27.200772+0000","last_became_active":"2024-09-16T10:33:17.861730+0000","last_became_peered":"2024-09-16T10:33:17.861730+0000","last_unstale":"2024-09-16T10:34:27.200772+0000","last_undegraded":"2024-09-16T10:34:27.200772+0000","last_fullsized":"2024-09-16T10:34:27.200772+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:58.055390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405233+0000","last_change":"2024-09-16T10:33:19.878150+0000","last_active":"2024-09-16T10:33:24.405233+0000","last_peered":"2024-09-16T10:33:24.405233+0000","last_clean":"2024-09-16T10:33:24.405233+0000","last_became_active":"2024-09-16T10:33:19.877676+0000","last_became_peered":"2024-09-16T10:33:19.877676+0000","last_unstale":"2024-09-16T10:33:24.405233+0000","last_undegraded":"2024-09-16T10:33:24.405233+0000","last_fullsized":"2024-09-16T10:33:24.405233+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:32:21.719673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117757+0000","last_change":"2024-09-16T10:33:15.844211+0000","last_active":"2024-09-16T10:33:24.117757+0000","last_peered":"2024-09-16T10:33:24.117757+0000","last_clean":"2024-09-16T10:33:24.117757+0000","last_became_active":"2024-09-16T10:33:15.843639+0000","last_became_peered":"2024-09-16T10:33:15.843639+0000","last_unstale":"2024-09-16T10:33:24.117757+0000","last_undegraded":"2024-09-16T10:33:24.117757+0000","last_fullsized":"2024-09-16T10:33:24.117757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:32.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404795+0000","last_change":"2024-09-16T10:33:13.833644+0000","last_active":"2024-09-16T10:33:24.404795+0000","last_peered":"2024-09-16T10:33:24.404795+0000","last_clean":"2024-09-16T10:33:24.404795+0000","last_became_active":"2024-09-16T10:33:13.833324+0000","last_became_peered":"2024-09-16T10:33:13.833324+0000","last_unstale":"2024-09-16T10:33:24.404795+0000","last_undegraded":"2024-09-16T10:33:24.404795+0000","last_fullsized":"2024-09-16T10:33:24.404795+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:47:36.327924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200889+0000","last_change":"2024-09-16T10:33:17.863472+0000","last_active":"2024-09-16T10:34:27.200889+0000","last_peered":"2024-09-16T10:34:27.200889+0000","last_clean":"2024-09-16T10:34:27.200889+0000","last_became_active":"2024-09-16T10:33:17.863190+0000","last_became_peered":"2024-09-16T10:33:17.863190+0000","last_unstale":"2024-09-16T10:34:27.200889+0000","last_undegraded":"2024-09-16T10:34:27.200889+0000","last_fullsized":"2024-09-16T10:34:27.200889+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:42:54.389460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404601+0000","last_change":"2024-09-16T10:33:19.869175+0000","last_active":"2024-09-16T10:33:24.404601+0000","last_peered":"2024-09-16T10:33:24.404601+0000","last_clean":"2024-09-16T10:33:24.404601+0000","last_became_active":"2024-09-16T10:33:19.868882+0000","last_became_peered":"2024-09-16T10:33:19.868882+0000","last_unstale":"2024-09-16T10:33:24.404601+0000","last_undegraded":"2024-09-16T10:33:24.404601+0000","last_fullsized":"2024-09-16T10:33:24.404601+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:16.217936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":33,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120901+0000","last_change":"2024-09-16T10:33:15.840633+0000","last_active":"2024-09-16T10:33:24.120901+0000","last_peered":"2024-09-16T10:33:24.120901+0000","last_clean":"2024-09-16T10:33:24.120901+0000","last_became_active":"2024-09-16T10:33:15.840347+0000","last_became_peered":"2024-09-16T10:33:15.840347+0000","last_unstale":"2024-09-16T10:33:24.120901+0000","last_undegraded":"2024-09-16T10:33:24.120901+0000","last_fullsized":"2024-09-16T10:33:24.120901+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:32.306172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403777+0000","last_change":"2024-09-16T10:33:13.830313+0000","last_active":"2024-09-16T10:33:24.403777+0000","last_peered":"2024-09-16T10:33:24.403777+0000","last_clean":"2024-09-16T10:33:24.403777+0000","last_became_active":"2024-09-16T10:33:13.830053+0000","last_became_peered":"2024-09-16T10:33:13.830053+0000","last_unstale":"2024-09-16T10:33:24.403777+0000","last_undegraded":"2024-09-16T10:33:24.403777+0000","last_fullsized":"2024-09-16T10:33:24.403777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:33.961850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121656+0000","last_change":"2024-09-16T10:33:17.861748+0000","last_active":"2024-09-16T10:33:24.121656+0000","last_peered":"2024-09-16T10:33:24.121656+0000","last_clean":"2024-09-16T10:33:24.121656+0000","last_became_active":"2024-09-16T10:33:17.861448+0000","last_became_peered":"2024-09-16T10:33:17.861448+0000","last_unstale":"2024-09-16T10:33:24.121656+0000","last_undegraded":"2024-09-16T10:33:24.121656+0000","last_fullsized":"2024-09-16T10:33:24.121656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:36.398920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403724+0000","last_change":"2024-09-16T10:33:19.877338+0000","last_active":"2024-09-16T10:33:24.403724+0000","last_peered":"2024-09-16T10:33:24.403724+0000","last_clean":"2024-09-16T10:33:24.403724+0000","last_became_active":"2024-09-16T10:33:19.875625+0000","last_became_peered":"2024-09-16T10:33:19.875625+0000","last_unstale":"2024-09-16T10:33:24.403724+0000","last_undegraded":"2024-09-16T10:33:24.403724+0000","last_fullsized":"2024-09-16T10:33:24.403724+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:40.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404042+0000","last_change":"2024-09-16T10:33:15.847097+0000","last_active":"2024-09-16T10:33:24.404042+0000","last_peered":"2024-09-16T10:33:24.404042+0000","last_clean":"2024-09-16T10:33:24.404042+0000","last_became_active":"2024-09-16T10:33:15.845338+0000","last_became_peered":"2024-09-16T10:33:15.845338+0000","last_unstale":"2024-09-16T10:33:24.404042+0000","last_undegraded":"2024-09-16T10:33:24.404042+0000","last_fullsized":"2024-09-16T10:33:24.404042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:19.356069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121935+0000","last_change":"2024-09-16T10:33:13.826300+0000","last_active":"2024-09-16T10:33:24.121935+0000","last_peered":"2024-09-16T10:33:24.121935+0000","last_clean":"2024-09-16T10:33:24.121935+0000","last_became_active":"2024-09-16T10:33:13.826015+0000","last_became_peered":"2024-09-16T10:33:13.826015+0000","last_unstale":"2024-09-16T10:33:24.121935+0000","last_undegraded":"2024-09-16T10:33:24.121935+0000","last_fullsized":"2024-09-16T10:33:24.121935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:57.183144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120726+0000","last_change":"2024-09-16T10:33:17.861980+0000","last_active":"2024-09-16T10:33:24.120726+0000","last_peered":"2024-09-16T10:33:24.120726+0000","last_clean":"2024-09-16T10:33:24.120726+0000","last_became_active":"2024-09-16T10:33:17.861704+0000","last_became_peered":"2024-09-16T10:33:17.861704+0000","last_unstale":"2024-09-16T10:33:24.120726+0000","last_undegraded":"2024-09-16T10:33:24.120726+0000","last_fullsized":"2024-09-16T10:33:24.120726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:55.118484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120763+0000","last_change":"2024-09-16T10:33:19.877272+0000","last_active":"2024-09-16T10:33:24.120763+0000","last_peered":"2024-09-16T10:33:24.120763+0000","last_clean":"2024-09-16T10:33:24.120763+0000","last_became_active":"2024-09-16T10:33:19.876986+0000","last_became_peered":"2024-09-16T10:33:19.876986+0000","last_unstale":"2024-09-16T10:33:24.120763+0000","last_undegraded":"2024-09-16T10:33:24.120763+0000","last_fullsized":"2024-09-16T10:33:24.120763+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:44:40.314655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121463+0000","last_change":"2024-09-16T10:33:15.848107+0000","last_active":"2024-09-16T10:33:24.121463+0000","last_peered":"2024-09-16T10:33:24.121463+0000","last_clean":"2024-09-16T10:33:24.121463+0000","last_became_active":"2024-09-16T10:33:15.847695+0000","last_became_peered":"2024-09-16T10:33:15.847695+0000","last_unstale":"2024-09-16T10:33:24.121463+0000","last_undegraded":"2024-09-16T10:33:24.121463+0000","last_fullsized":"2024-09-16T10:33:24.121463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:27:51.896174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117814+0000","last_change":"2024-09-16T10:33:13.828318+0000","last_active":"2024-09-16T10:33:24.117814+0000","last_peered":"2024-09-16T10:33:24.117814+0000","last_clean":"2024-09-16T10:33:24.117814+0000","last_became_active":"2024-09-16T10:33:13.828064+0000","last_became_peered":"2024-09-16T10:33:13.828064+0000","last_unstale":"2024-09-16T10:33:24.117814+0000","last_undegraded":"2024-09-16T10:33:24.117814+0000","last_fullsized":"2024-09-16T10:33:24.117814+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:38.053804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404153+0000","last_change":"2024-09-16T10:33:17.862953+0000","last_active":"2024-09-16T10:33:24.404153+0000","last_peered":"2024-09-16T10:33:24.404153+0000","last_clean":"2024-09-16T10:33:24.404153+0000","last_became_active":"2024-09-16T10:33:17.862706+0000","last_became_peered":"2024-09-16T10:33:17.862706+0000","last_unstale":"2024-09-16T10:33:24.404153+0000","last_undegraded":"2024-09-16T10:33:24.404153+0000","last_fullsized":"2024-09-16T10:33:24.404153+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:13.727989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403643+0000","last_change":"2024-09-16T10:33:19.870777+0000","last_active":"2024-09-16T10:33:24.403643+0000","last_peered":"2024-09-16T10:33:24.403643+0000","last_clean":"2024-09-16T10:33:24.403643+0000","last_became_active":"2024-09-16T10:33:19.870479+0000","last_became_peered":"2024-09-16T10:33:19.870479+0000","last_unstale":"2024-09-16T10:33:24.403643+0000","last_undegraded":"2024-09-16T10:33:24.403643+0000","last_fullsized":"2024-09-16T10:33:24.403643+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:04.103959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120983+0000","last_change":"2024-09-16T10:33:15.848126+0000","last_active":"2024-09-16T10:33:24.120983+0000","last_peered":"2024-09-16T10:33:24.120983+0000","last_clean":"2024-09-16T10:33:24.120983+0000","last_became_active":"2024-09-16T10:33:15.847882+0000","last_became_peered":"2024-09-16T10:33:15.847882+0000","last_unstale":"2024-09-16T10:33:24.120983+0000","last_undegraded":"2024-09-16T10:33:24.120983+0000","last_fullsized":"2024-09-16T10:33:24.120983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:57.050068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404154+0000","last_change":"2024-09-16T10:33:13.824934+0000","last_active":"2024-09-16T10:33:24.404154+0000","last_peered":"2024-09-16T10:33:24.404154+0000","last_clean":"2024-09-16T10:33:24.404154+0000","last_became_active":"2024-09-16T10:33:13.824667+0000","last_became_peered":"2024-09-16T10:33:13.824667+0000","last_unstale":"2024-09-16T10:33:24.404154+0000","last_undegraded":"2024-09-16T10:33:24.404154+0000","last_fullsized":"2024-09-16T10:33:24.404154+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:56.178015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405376+0000","last_change":"2024-09-16T10:33:17.860036+0000","last_active":"2024-09-16T10:33:24.405376+0000","last_peered":"2024-09-16T10:33:24.405376+0000","last_clean":"2024-09-16T10:33:24.405376+0000","last_became_active":"2024-09-16T10:33:17.859392+0000","last_became_peered":"2024-09-16T10:33:17.859392+0000","last_unstale":"2024-09-16T10:33:24.405376+0000","last_undegraded":"2024-09-16T10:33:24.405376+0000","last_fullsized":"2024-09-16T10:33:24.405376+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:17.305923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121492+0000","last_change":"2024-09-16T10:33:19.909507+0000","last_active":"2024-09-16T10:33:24.121492+0000","last_peered":"2024-09-16T10:33:24.121492+0000","last_clean":"2024-09-16T10:33:24.121492+0000","last_became_active":"2024-09-16T10:33:19.909204+0000","last_became_peered":"2024-09-16T10:33:19.909204+0000","last_unstale":"2024-09-16T10:33:24.121492+0000","last_undegraded":"2024-09-16T10:33:24.121492+0000","last_fullsized":"2024-09-16T10:33:24.121492+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:44.376563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403866+0000","last_change":"2024-09-16T10:33:19.872836+0000","last_active":"2024-09-16T10:33:24.403866+0000","last_peered":"2024-09-16T10:33:24.403866+0000","last_clean":"2024-09-16T10:33:24.403866+0000","last_became_active":"2024-09-16T10:33:19.872554+0000","last_became_peered":"2024-09-16T10:33:19.872554+0000","last_unstale":"2024-09-16T10:33:24.403866+0000","last_undegraded":"2024-09-16T10:33:24.403866+0000","last_fullsized":"2024-09-16T10:33:24.403866+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:08:13.249106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.117246+0000","last_change":"2024-09-16T10:33:13.831366+0000","last_active":"2024-09-16T10:33:24.117246+0000","last_peered":"2024-09-16T10:33:24.117246+0000","last_clean":"2024-09-16T10:33:24.117246+0000","last_became_active":"2024-09-16T10:33:13.831045+0000","last_became_peered":"2024-09-16T10:33:13.831045+0000","last_unstale":"2024-09-16T10:33:24.117246+0000","last_undegraded":"2024-09-16T10:33:24.117246+0000","last_fullsized":"2024-09-16T10:33:24.117246+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:11.658752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":32,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405298+0000","last_change":"2024-09-16T10:33:15.854039+0000","last_active":"2024-09-16T10:33:24.405298+0000","last_peered":"2024-09-16T10:33:24.405298+0000","last_clean":"2024-09-16T10:33:24.405298+0000","last_became_active":"2024-09-16T10:33:15.853711+0000","last_became_peered":"2024-09-16T10:33:15.853711+0000","last_unstale":"2024-09-16T10:33:24.405298+0000","last_undegraded":"2024-09-16T10:33:24.405298+0000","last_fullsized":"2024-09-16T10:33:24.405298+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:26.060755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T10:34:27.200733+0000","last_change":"2024-09-16T10:33:17.866228+0000","last_active":"2024-09-16T10:34:27.200733+0000","last_peered":"2024-09-16T10:34:27.200733+0000","last_clean":"2024-09-16T10:34:27.200733+0000","last_became_active":"2024-09-16T10:33:17.865902+0000","last_became_peered":"2024-09-16T10:33:17.865902+0000","last_unstale":"2024-09-16T10:34:27.200733+0000","last_undegraded":"2024-09-16T10:34:27.200733+0000","last_fullsized":"2024-09-16T10:34:27.200733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:26.959193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.120627+0000","last_change":"2024-09-16T10:33:19.870399+0000","last_active":"2024-09-16T10:33:24.120627+0000","last_peered":"2024-09-16T10:33:24.120627+0000","last_clean":"2024-09-16T10:33:24.120627+0000","last_became_active":"2024-09-16T10:33:19.870110+0000","last_became_peered":"2024-09-16T10:33:19.870110+0000","last_unstale":"2024-09-16T10:33:24.120627+0000","last_undegraded":"2024-09-16T10:33:24.120627+0000","last_fullsized":"2024-09-16T10:33:24.120627+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:53:10.134858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121303+0000","last_change":"2024-09-16T10:33:13.831459+0000","last_active":"2024-09-16T10:33:24.121303+0000","last_peered":"2024-09-16T10:33:24.121303+0000","last_clean":"2024-09-16T10:33:24.121303+0000","last_became_active":"2024-09-16T10:33:13.831054+0000","last_became_peered":"2024-09-16T10:33:13.831054+0000","last_unstale":"2024-09-16T10:33:24.121303+0000","last_undegraded":"2024-09-16T10:33:24.121303+0000","last_fullsized":"2024-09-16T10:33:24.121303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:26.290480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":36,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404282+0000","last_change":"2024-09-16T10:33:15.843213+0000","last_active":"2024-09-16T10:33:24.404282+0000","last_peered":"2024-09-16T10:33:24.404282+0000","last_clean":"2024-09-16T10:33:24.404282+0000","last_became_active":"2024-09-16T10:33:15.842893+0000","last_became_peered":"2024-09-16T10:33:15.842893+0000","last_unstale":"2024-09-16T10:33:24.404282+0000","last_undegraded":"2024-09-16T10:33:24.404282+0000","last_fullsized":"2024-09-16T10:33:24.404282+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:48.208043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405295+0000","last_change":"2024-09-16T10:33:17.858923+0000","last_active":"2024-09-16T10:33:24.405295+0000","last_peered":"2024-09-16T10:33:24.405295+0000","last_clean":"2024-09-16T10:33:24.405295+0000","last_became_active":"2024-09-16T10:33:17.858605+0000","last_became_peered":"2024-09-16T10:33:17.858605+0000","last_unstale":"2024-09-16T10:33:24.405295+0000","last_undegraded":"2024-09-16T10:33:24.405295+0000","last_fullsized":"2024-09-16T10:33:24.405295+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:10:21.376810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405106+0000","last_change":"2024-09-16T10:33:19.908026+0000","last_active":"2024-09-16T10:33:24.405106+0000","last_peered":"2024-09-16T10:33:24.405106+0000","last_clean":"2024-09-16T10:33:24.405106+0000","last_became_active":"2024-09-16T10:33:19.907758+0000","last_became_peered":"2024-09-16T10:33:19.907758+0000","last_unstale":"2024-09-16T10:33:24.405106+0000","last_undegraded":"2024-09-16T10:33:24.405106+0000","last_fullsized":"2024-09-16T10:33:24.405106+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:20.958848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.405422+0000","last_change":"2024-09-16T10:33:13.820521+0000","last_active":"2024-09-16T10:33:24.405422+0000","last_peered":"2024-09-16T10:33:24.405422+0000","last_clean":"2024-09-16T10:33:24.405422+0000","last_became_active":"2024-09-16T10:33:13.820193+0000","last_became_peered":"2024-09-16T10:33:13.820193+0000","last_unstale":"2024-09-16T10:33:24.405422+0000","last_undegraded":"2024-09-16T10:33:24.405422+0000","last_fullsized":"2024-09-16T10:33:24.405422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:50.955174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.404487+0000","last_change":"2024-09-16T10:33:15.842037+0000","last_active":"2024-09-16T10:33:24.404487+0000","last_peered":"2024-09-16T10:33:24.404487+0000","last_clean":"2024-09-16T10:33:24.404487+0000","last_became_active":"2024-09-16T10:33:15.841722+0000","last_became_peered":"2024-09-16T10:33:15.841722+0000","last_unstale":"2024-09-16T10:33:24.404487+0000","last_undegraded":"2024-09-16T10:33:24.404487+0000","last_fullsized":"2024-09-16T10:33:24.404487+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:27.128996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121778+0000","last_change":"2024-09-16T10:33:17.859305+0000","last_active":"2024-09-16T10:33:24.121778+0000","last_peered":"2024-09-16T10:33:24.121778+0000","last_clean":"2024-09-16T10:33:24.121778+0000","last_became_active":"2024-09-16T10:33:17.858981+0000","last_became_peered":"2024-09-16T10:33:17.858981+0000","last_unstale":"2024-09-16T10:33:24.121778+0000","last_undegraded":"2024-09-16T10:33:24.121778+0000","last_fullsized":"2024-09-16T10:33:24.121778+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:10.081563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.121653+0000","last_change":"2024-09-16T10:33:19.908120+0000","last_active":"2024-09-16T10:33:24.121653+0000","last_peered":"2024-09-16T10:33:24.121653+0000","last_clean":"2024-09-16T10:33:24.121653+0000","last_became_active":"2024-09-16T10:33:19.907807+0000","last_became_peered":"2024-09-16T10:33:19.907807+0000","last_unstale":"2024-09-16T10:33:24.121653+0000","last_undegraded":"2024-09-16T10:33:24.121653+0000","last_fullsized":"2024-09-16T10:33:24.121653+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:18.838136+0000","last_clean_scrub_stamp":"2024-09-16T10:33:18.838136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:23.684854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122075+0000","last_change":"2024-09-16T10:33:13.823143+0000","last_active":"2024-09-16T10:33:24.122075+0000","last_peered":"2024-09-16T10:33:24.122075+0000","last_clean":"2024-09-16T10:33:24.122075+0000","last_became_active":"2024-09-16T10:33:13.822870+0000","last_became_peered":"2024-09-16T10:33:13.822870+0000","last_unstale":"2024-09-16T10:33:24.122075+0000","last_undegraded":"2024-09-16T10:33:24.122075+0000","last_fullsized":"2024-09-16T10:33:24.122075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:12.798040+0000","last_clean_scrub_stamp":"2024-09-16T10:33:12.798040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:46.899315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":40,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.122510+0000","last_change":"2024-09-16T10:33:15.848076+0000","last_active":"2024-09-16T10:33:24.122510+0000","last_peered":"2024-09-16T10:33:24.122510+0000","last_clean":"2024-09-16T10:33:24.122510+0000","last_became_active":"2024-09-16T10:33:15.847791+0000","last_became_peered":"2024-09-16T10:33:15.847791+0000","last_unstale":"2024-09-16T10:33:24.122510+0000","last_undegraded":"2024-09-16T10:33:24.122510+0000","last_fullsized":"2024-09-16T10:33:24.122510+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:14.811467+0000","last_clean_scrub_stamp":"2024-09-16T10:33:14.811467+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:17.088207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T10:33:24.403950+0000","last_change":"2024-09-16T10:33:17.865501+0000","last_active":"2024-09-16T10:33:24.403950+0000","last_peered":"2024-09-16T10:33:24.403950+0000","last_clean":"2024-09-16T10:33:24.403950+0000","last_became_active":"2024-09-16T10:33:17.865169+0000","last_became_peered":"2024-09-16T10:33:17.865169+0000","last_unstale":"2024-09-16T10:33:24.403950+0000","last_undegraded":"2024-09-16T10:33:24.403950+0000","last_fullsized":"2024-09-16T10:33:24.403950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T10:33:16.826716+0000","last_clean_scrub_stamp":"2024-09-16T10:33:16.826716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:23.752283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1386,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":333,"num_write":245,"num_write_kb":3982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201303,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32452,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708604,"statfs":{"total":95990841344,"available":95957610496,"internally_reserved":0,"allocated":1372160,"data_stored":911333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.505}]},{"osd":1,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47999999999999998}]},{"osd":2,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.495},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57399999999999995}]},{"osd":3,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54700000000000004}]},{"osd":4,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.377}]},{"osd":6,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.443},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.45400000000000001}]}]},{"osd":6,"up_from":47,"seq":201863462940,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32428,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708628,"statfs":{"total":95990841344,"available":95957635072,"internally_reserved":0,"allocated":1347584,"data_stored":909551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.621},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.53500000000000003}]},{"osd":1,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.59399999999999997}]},{"osd":3,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.35899999999999999}]},{"osd":5,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.376},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.442}]},{"osd":7,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.505}]}]},{"osd":5,"up_from":39,"seq":167503724576,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713292,"statfs":{"total":95990841344,"available":95962411008,"internally_reserved":0,"allocated":761856,"data_stored":319397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.503,"15min":0.499},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.66200000000000003}]},{"osd":1,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.48999999999999999}]},{"osd":2,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63800000000000001}]},{"osd":3,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79500000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.503}]},{"osd":4,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38200000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.29799999999999999}]},{"osd":6,"last update":"Mon Sep 16 10:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.375}]},{"osd":7,"last update":"Mon Sep 16 10:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.36099999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713280,"statfs":{"total":95990841344,"available":95962398720,"internally_reserved":0,"allocated":774144,"data_stored":319522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.28499999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.59899999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.313},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.38500000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60699999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.29299999999999998}]},{"osd":1,"last update":"Mon Sep 16 10:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.56699999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.308,"15min":0.308},"max":{"1min":0.65100000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.35299999999999998}]},{"osd":2,"last update":"Mon Sep 16 10:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52600000000000002,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61499999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.52500000000000002,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.622,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.378}]},{"osd":3,"last update":"Mon Sep 16 10:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66200000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Mon Sep 16 10:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.42599999999999999,"15min":0.41599999999999998},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66800000000000004,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.49099999999999999}]},{"osd":6,"last update":"Mon Sep 16 10:33:36 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.46999999999999997}]},{"osd":7,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.53800000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713256,"statfs":{"total":95990841344,"available":95962374144,"internally_reserved":0,"allocated":798720,"data_stored":320364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53700000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.434,"15min":0.44500000000000001},"min":{"1min":0.32700000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.48299999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.50800000000000001}]},{"osd":1,"last update":"Mon Sep 16 10:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55800000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41699999999999998,"15min":0.42299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69399999999999995,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.432}]},{"osd":2,"last update":"Mon Sep 16 10:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42799999999999999,"15min":0.434},"min":{"1min":0.29999999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58599999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.46600000000000003,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57499999999999996,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.45100000000000001}]},{"osd":4,"last update":"Mon Sep 16 10:33:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Sep 16 10:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90100000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57399999999999995}]},{"osd":6,"last update":"Mon Sep 16 10:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.628}]},{"osd":7,"last update":"Mon Sep 16 10:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60099999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713292,"statfs":{"total":95990841344,"available":95962411008,"internally_reserved":0,"allocated":761856,"data_stored":320263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35699999999999998,"15min":0.36099999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.502,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.379},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.374,"15min":0.379},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.27100000000000002}]},{"osd":1,"last update":"Mon Sep 16 10:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.377,"15min":0.378},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52800000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52400000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.40999999999999998}]},{"osd":3,"last update":"Mon Sep 16 10:34:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.41199999999999998,"15min":0.42799999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53800000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.441,"15min":0.45500000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59599999999999997,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.32900000000000001}]},{"osd":4,"last update":"Mon Sep 16 10:33:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.38,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.39100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Sep 16 10:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75900000000000001,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.69099999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Sep 16 10:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73999999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71099999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.5}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713240,"statfs":{"total":95990841344,"available":95962357760,"internally_reserved":0,"allocated":815104,"data_stored":321628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 10:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.316},"min":{"1min":0.28199999999999997,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.47999999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.33700000000000002,"15min":0.316},"min":{"1min":0.28999999999999998,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.307}]},{"osd":2,"last update":"Mon Sep 16 10:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33100000000000002,"15min":0.32500000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.374,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50900000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.35599999999999998}]},{"osd":3,"last update":"Mon Sep 16 10:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42299999999999999,"15min":0.42899999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.56299999999999994,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.42299999999999999},"min":{"1min":0.27800000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59499999999999997,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.434}]},{"osd":4,"last update":"Mon Sep 16 10:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71299999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40500000000000003}]},{"osd":5,"last update":"Mon Sep 16 10:34:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54100000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.42899999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68899999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41999999999999998}]},{"osd":6,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46300000000000002}]},{"osd":7,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.374}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32452,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93708604,"statfs":{"total":95990841344,"available":95957610496,"internally_reserved":0,"allocated":1372160,"data_stored":910941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 10:33:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.34899999999999998,"15min":0.34300000000000003},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.38100000000000001,"15min":0.36499999999999999},"min":{"1min":0.27500000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.373}]},{"osd":2,"last update":"Mon Sep 16 10:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.51800000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36099999999999999}]},{"osd":3,"last update":"Mon Sep 16 10:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.437,"15min":0.442},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.495,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.316}]},{"osd":4,"last update":"Mon Sep 16 10:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.38100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.51100000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.73299999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56200000000000006}]},{"osd":5,"last update":"Mon Sep 16 10:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.46400000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54000000000000004}]},{"osd":6,"last update":"Mon Sep 16 10:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.40300000000000002}]},{"osd":7,"last update":"Mon Sep 16 10:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.42999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T10:34:39.432 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T10:34:39.433 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T10:34:39.433 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T10:34:39.433 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T10:34:39.631 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:40.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:34:40 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug there is no tcmu-runner data available 2024-09-16T10:34:40.426 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:40 smithi164 ceph-mon[40346]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T10:34:40.426 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:40 smithi164 ceph-mon[40346]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:40.426 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:40 smithi164 ceph-mon[25828]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T10:34:40.426 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:40 smithi164 ceph-mon[25828]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:40.427 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:34:40 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[83411]: ts=2024-09-16T10:34:40.263Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002977356s 2024-09-16T10:34:40.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:40 smithi175 ceph-mon[30341]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T10:34:40.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:40 smithi175 ceph-mon[30341]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:41.151 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 systemd[1]: Starting Ceph grafana.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:34:41.423 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-mon[30341]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:41.423 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 podman[70920]: 2024-09-16 10:34:41.210044474 +0000 UTC m=+0.015095749 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-16T10:34:41.423 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 podman[70920]: 2024-09-16 10:34:41.3118311 +0000 UTC m=+0.116882361 container create bada0a3c81e0a09572a613b9082cad98bb49b2c77db0369fce2ff09d6f09a2e3 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a, vcs-type=git, vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , description=Ceph Grafana Container, name=ubi8, release=239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, io.openshift.expose-services=, io.openshift.tags=base rhel8, architecture=x86_64, com.redhat.component=ubi8-container, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0) 2024-09-16T10:34:41.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:41 smithi164 ceph-mon[40346]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:41.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:41 smithi164 ceph-mon[25828]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T10:34:41.480 INFO:teuthology.orchestra.run.smithi164.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T10:34:41.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 podman[70920]: 2024-09-16 10:34:41.547577151 +0000 UTC m=+0.352628417 container init bada0a3c81e0a09572a613b9082cad98bb49b2c77db0369fce2ff09d6f09a2e3 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, vcs-type=git, description=Ceph Grafana Container, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, distribution-scope=public, maintainer=Paul Cuzner , com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, com.redhat.component=ubi8-container, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.tags=base rhel8, architecture=x86_64) 2024-09-16T10:34:41.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 podman[70920]: 2024-09-16 10:34:41.552150022 +0000 UTC m=+0.357201280 container start bada0a3c81e0a09572a613b9082cad98bb49b2c77db0369fce2ff09d6f09a2e3 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a, maintainer=Paul Cuzner , io.openshift.expose-services=, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, architecture=x86_64, release=239.1651231664, name=ubi8, version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, distribution-scope=public, io.buildah.version=1.32.0, description=Ceph Grafana Container, vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git) 2024-09-16T10:34:41.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 bash[70920]: bada0a3c81e0a09572a613b9082cad98bb49b2c77db0369fce2ff09d6f09a2e3 2024-09-16T10:34:41.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 systemd[1]: Started Ceph grafana.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:34:41.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706576509Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-16T10:34:41.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706749497Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.70676814Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706775797Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706781909Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706787785Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706793965Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706801034Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706806849Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706812866Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706818247Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706823499Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706829238Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-16T10:34:41.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=settings t=2024-09-16T10:34:41.706834924Z level=info msg="App mode production" 2024-09-16T10:34:41.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=sqlstore t=2024-09-16T10:34:41.706879448Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-16T10:34:41.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=sqlstore t=2024-09-16T10:34:41.706895668Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-16T10:34:41.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.707738806Z level=info msg="Starting DB migrations" 2024-09-16T10:34:41.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.707810381Z level=info msg="Executing migration" id="create migration_log table" 2024-09-16T10:34:41.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.708317042Z level=info msg="Executing migration" id="create user table" 2024-09-16T10:34:41.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.708723534Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-16T10:34:41.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.709560924Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-16T10:34:41.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.710408631Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-16T10:34:41.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.711241818Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-16T10:34:41.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.71208499Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-16T10:34:41.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.712948993Z level=info msg="Executing migration" id="create user table v2" 2024-09-16T10:34:41.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.713785744Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-16T10:34:41.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.714653168Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-16T10:34:41.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.715435392Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.715779804Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.716755922Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.717481135Z level=info msg="Executing migration" id="Update user table charset" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.717771436Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.718191007Z level=info msg="Executing migration" id="Add missing user data" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.718533376Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.718921598Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.719337409Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.719717922Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.720807965Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.721246156Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.721664538Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-16T10:34:41.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.722056137Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.722463765Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.722839067Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.723130995Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.723498661Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.723844563Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.724222066Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.724590483Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.725209071Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.725594726Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.725952151Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.726366658Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.726707429Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-16T10:34:41.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.727085564Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.727406931Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.72770893Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.728009939Z level=info msg="Executing migration" id="create star table" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.728367285Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.728764608Z level=info msg="Executing migration" id="create org table v1" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.729150916Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.729541897Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.729890867Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.730286806Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.730733564Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.731195626Z level=info msg="Executing migration" id="Update org table charset" 2024-09-16T10:34:41.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.731491313Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.731753517Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.732043981Z level=info msg="Executing migration" id="create dashboard table" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.732452687Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.733244684Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.733664836Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.734052004Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.734924669Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.735761916Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.73678426Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.737621778Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.738621532Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.739341477Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.740057164Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-16T10:34:41.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.740466477Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-16T10:34:41.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.740733571Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-16T10:34:41.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.741324412Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-16T10:34:41.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.742293993Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-16T10:34:41.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.743048625Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-16T10:34:41.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.744033045Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-16T10:34:41.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.744750565Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-16T10:34:41.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.745771916Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-16T10:34:41.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.746551996Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-16T10:34:41.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.746839794Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.747126068Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.747673979Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.748526599Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.749393513Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.750334988Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.751094996Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.751957171Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.752937163Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.753625246Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.754593048Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.755134241Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.756061293Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-16T10:34:41.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.758981536Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.761618577Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.764667512Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.767253843Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.77020602Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.773044184Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.775830176Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.77852321Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.78122417Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.784030817Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.786867231Z level=info msg="Executing migration" id="create data_source table" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.789767025Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.790214448Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-16T10:34:41.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.792512455Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-16T10:34:41.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.795297275Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-16T10:34:41.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.798099669Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-16T10:34:42.051 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.800938559Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-16T10:34:42.051 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.803683727Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-16T10:34:42.051 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.806513306Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-16T10:34:42.051 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.809261964Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-16T10:34:42.051 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.812046873Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-16T10:34:42.051 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.814559655Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-16T10:34:42.051 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.817272912Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-16T10:34:42.051 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.819800308Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-16T10:34:42.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.822774529Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-16T10:34:42.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.825198268Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-16T10:34:42.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.827856714Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-16T10:34:42.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.830614044Z level=info msg="Executing migration" id="Add uid column" 2024-09-16T10:34:42.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.833220184Z level=info msg="Executing migration" id="Update uid value" 2024-09-16T10:34:42.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.835941253Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-16T10:34:42.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.838691808Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-16T10:34:42.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.841332863Z level=info msg="Executing migration" id="create api_key table" 2024-09-16T10:34:42.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.843886383Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-16T10:34:42.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.846667246Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-16T10:34:42.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.849244257Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-16T10:34:42.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.852028852Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-16T10:34:42.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.852774533Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-16T10:34:42.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.853626572Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-16T10:34:42.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.854511766Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-16T10:34:42.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.855558298Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-16T10:34:42.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.855950606Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-16T10:34:42.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.856852364Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-16T10:34:42.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.857821731Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-16T10:34:42.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.858628788Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-16T10:34:42.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.858995707Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-16T10:34:42.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.859368871Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-16T10:34:42.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.860194839Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.860779645Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.861343734Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.861670662Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.862538691Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.863536303Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.864208344Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.864636516Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.865456558Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.865915915Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.866362444Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.867215Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.867566893Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-16T10:34:42.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.868406088Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.869005555Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.869613473Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.870502617Z level=info msg="Executing migration" id="create quota table v1" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.871002144Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.871448884Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.871736031Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.872300552Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.87313208Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.874021479Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.874336734Z level=info msg="Executing migration" id="create session table" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.875214016Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.876188969Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-16T10:34:42.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.876975903Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.877377296Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.878340216Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.87914929Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.879988106Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.880287965Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.881124143Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.881532053Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.88194634Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.882577592Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.882874936Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.883508523Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.884112589Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.884950055Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-16T10:34:42.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.885916958Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.886663821Z level=info msg="Executing migration" id="create alert table v1" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.887148671Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.890059452Z level=info msg="Executing migration" id="add index alert state" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.892889521Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.895679293Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.896153138Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.89891757Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.901724706Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.904700142Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.90508882Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.908164178Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.910831152Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.913793209Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-16T10:34:42.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.914190492Z level=info msg="Executing migration" id="Add column is_default" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.91699662Z level=info msg="Executing migration" id="Add column frequency" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.919797286Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.92240158Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.923056709Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.925926448Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.928641875Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.931370858Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.931765388Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.934727258Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.937412927Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.940038478Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.940436965Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-16T10:34:42.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.943221368Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.943928518Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.945946614Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.948709783Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.949138056Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.95192408Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.954865916Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.957592769Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.95807532Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.960715242Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.963524877Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.964310679Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.965122008Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.96606149Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-16T10:34:42.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.966793802Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.967638466Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.968088512Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.968949598Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.969911443Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.970732205Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.971127335Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.971946498Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.972903521Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.974580966Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.974970089Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.975829206Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.976641679Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-16T10:34:42.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.977536791Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-16T10:34:42.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.977868915Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-16T10:34:42.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.978743125Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-16T10:34:42.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.979593471Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-16T10:34:42.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.980591698Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-16T10:34:42.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.981065764Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-16T10:34:42.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.982043063Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-16T10:34:42.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.982861976Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-16T10:34:42.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.983797907Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-16T10:34:42.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.984162281Z level=info msg="Executing migration" id="Move region to single row" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.985046329Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.985998717Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.986834948Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.987499871Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.988395289Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.989032561Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.989488607Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.990325414Z level=info msg="Executing migration" id="create test_data table" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.990778224Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.991224933Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.994087305Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-16T10:34:42.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:41 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:41.996981297Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.000510412Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.002498814Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.00518487Z level=info msg="Executing migration" id="create team table" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.00798979Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.010815993Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.013659067Z level=info msg="Executing migration" id="create team member table" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.016516478Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.019106264Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.022024052Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.024716412Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.027677098Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.030365908Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-16T10:34:42.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.033325537Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-16T10:34:42.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.03620356Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-16T10:34:42.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.039034754Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-16T10:34:42.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.04181Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-16T10:34:42.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.044570262Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-16T10:34:42.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.04743143Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-16T10:34:42.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.049915807Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-16T10:34:42.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.052467892Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-16T10:34:42.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.052905976Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-16T10:34:42.147 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T10:34:42.147 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T10:34:42.147 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T10:34:42.159 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T10:34:42.160 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-09-16T10:34:42.301 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-mon[30341]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:42.301 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-mon[30341]: from='client.? 172.21.15.164:0/3483802172' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T10:34:42.301 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.301 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.301 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.301 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.055654645Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-16T10:34:42.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.058285822Z level=info msg="Executing migration" id="create tag table" 2024-09-16T10:34:42.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.061102551Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-16T10:34:42.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.061673094Z level=info msg="Executing migration" id="create login attempt table" 2024-09-16T10:34:42.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.064528176Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-16T10:34:42.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.067250223Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-16T10:34:42.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.069737915Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-16T10:34:42.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.071616756Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-16T10:34:42.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.074493419Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-16T10:34:42.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.077207352Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.079669661Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.080075679Z level=info msg="Executing migration" id="create user auth table" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.080883023Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.08182384Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.082542296Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.083338197Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.084115992Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.084895641Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.085686128Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.086135786Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.08893848Z level=info msg="Executing migration" id="create server_lock table" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.0917092Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-16T10:34:42.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.094642564Z level=info msg="Executing migration" id="create user auth token table" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.095228407Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.098155488Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.100666808Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.103595097Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.104423229Z level=info msg="Executing migration" id="create cache_data table" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.107064036Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.1099058Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.112507049Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.113145114Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.11585045Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.116198413Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.11667229Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-16T10:34:42.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.11911576Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.119536271Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.119795613Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.120200315Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.120598941Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.120993729Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.121478997Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.122265176Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.122706328Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.122979028Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.12341006Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.123798582Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.124213018Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.124496286Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-16T10:34:42.305 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.124941081Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.125481726Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.1258676Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.126286566Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.127053933Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.127458183Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.127828691Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.131186096Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.134496358Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.13492199Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.135463015Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.136332275Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.136735052Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-16T10:34:42.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.137170523Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-16T10:34:42.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.137585589Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-16T10:34:42.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.137998405Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-16T10:34:42.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.138292838Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-16T10:34:42.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.139085015Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-16T10:34:42.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.139893866Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-16T10:34:42.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.140717031Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-16T10:34:42.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.14114686Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-16T10:34:42.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.141711706Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-16T10:34:42.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.142650783Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-16T10:34:42.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.143488062Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-16T10:34:42.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.143932656Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-16T10:34:42.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.144766197Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-16T10:34:42.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.14556803Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.145868837Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.146310974Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.146730782Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.147162707Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.14745013Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.149185415Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.150428927Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.151331633Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.152249881Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.153100303Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.153426802Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.15384951Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-16T10:34:42.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.154681806Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.155151552Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.156065292Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.158696714Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.161389649Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.164067084Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.166758261Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.169176431Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.171701661Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.174470681Z level=info msg="Executing migration" id="create alert_image table" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.177262301Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.177988857Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.178745381Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.179190616Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-16T10:34:42.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.180028038Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.180130866Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.180949687Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.181686489Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.182641945Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.182842921Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.182994377Z level=info msg="alerts found to migrate" alerts=0 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.183417079Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.184286336Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.185067457Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.186103198Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.186780919Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.187757153Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-16T10:34:42.311 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.188481103Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.189477194Z level=info msg="Executing migration" id="create data_keys table" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.190181051Z level=info msg="Executing migration" id="create secrets table" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.191054741Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.195374177Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.196318336Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.197222857Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.201479069Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.2059463Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.206835174Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.207811612Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.208638647Z level=info msg="Executing migration" id="create permission table" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.209408158Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.210461303Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-16T10:34:42.312 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.211186327Z level=info msg="Executing migration" id="create role table" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.212087069Z level=info msg="Executing migration" id="add column display_name" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.213020132Z level=info msg="Executing migration" id="add column group_name" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.213921347Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.214804756Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.215628431Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.216686238Z level=info msg="Executing migration" id="create team role table" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.217439918Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.218367663Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.221119521Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.223821901Z level=info msg="Executing migration" id="create user role table" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.226793554Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.229473737Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.232315925Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-16T10:34:42.313 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.2351872Z level=info msg="Executing migration" id="create builtin role table" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.23787863Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.240707178Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.243396623Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.246055223Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.246540962Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.249524834Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.252123435Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.254890819Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.25744156Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.260223585Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.261240367Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.262234289Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-16T10:34:42.314 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.262910753Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.263890051Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.264578167Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.265406662Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.265911456Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.266213065Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.26720823Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.267930477Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.268774073Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.269762277Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.270564871Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.271406982Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.272238492Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.273020463Z level=info msg="Executing migration" id="add correlation config column" 2024-09-16T10:34:42.315 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.274025546Z level=info msg="Executing migration" id="create entity_events table" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.275006676Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.275906786Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.276163412Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.276614427Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.276692768Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.277495166Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.27808612Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.278507078Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.27937724Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.280214453Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.281067914Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.282049318Z level=info msg="Executing migration" id="Drop public config table" 2024-09-16T10:34:42.316 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.28278278Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-16T10:34:42.317 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.283680416Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-16T10:34:42.317 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.28450726Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T10:34:42.317 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.285316292Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-16T10:34:42.317 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.28617251Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-16T10:34:42.317 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.289219239Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-16T10:34:42.317 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.29055811Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-16T10:34:42.317 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.291560725Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-16T10:34:42.317 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.29452358Z level=info msg="Executing migration" id="add share column" 2024-09-16T10:34:42.317 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.297252381Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-16T10:34:42.317 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.300118751Z level=info msg="Executing migration" id="create file table" 2024-09-16T10:34:42.317 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.302767282Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-16T10:34:42.364 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:42.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[40346]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:42.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[40346]: from='client.? 172.21.15.164:0/3483802172' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T10:34:42.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[25828]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:42.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[25828]: from='client.? 172.21.15.164:0/3483802172' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T10:34:42.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:42 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:42.735 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.305633766Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-16T10:34:42.736 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.30844843Z level=info msg="Executing migration" id="create file_meta table" 2024-09-16T10:34:42.736 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.311272326Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-16T10:34:42.736 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.314082971Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-16T10:34:42.736 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.316877675Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-16T10:34:42.736 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.317367412Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-16T10:34:42.736 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.320077582Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-16T10:34:42.736 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.320516154Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-16T10:34:42.736 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.322871665Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-16T10:34:42.736 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.325816798Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-16T10:34:42.736 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.326314101Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-16T10:34:42.737 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.329323327Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-16T10:34:42.737 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.331989275Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-16T10:34:42.737 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.334824547Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-16T10:34:42.737 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.335890902Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-16T10:34:42.737 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.33863199Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-16T10:34:42.737 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.341450585Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-16T10:34:42.737 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.351499208Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-16T10:34:42.737 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.351946953Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-16T10:34:42.737 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.354785173Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-16T10:34:42.738 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.35835588Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-16T10:34:42.738 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.361105223Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-16T10:34:42.738 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.361540663Z level=info msg="Executing migration" id="create folder table" 2024-09-16T10:34:42.738 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.364251197Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-16T10:34:42.738 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.364696561Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-16T10:34:42.738 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.367033467Z level=info msg="Executing migration" id="Update folder title length" 2024-09-16T10:34:42.738 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.369716079Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-16T10:34:42.738 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=migrator t=2024-09-16T10:34:42.370241203Z level=info msg="migrations completed" performed=485 skipped=0 duration=662.449963ms 2024-09-16T10:34:42.738 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=sqlstore t=2024-09-16T10:34:42.370614143Z level=info msg="Created default organization" 2024-09-16T10:34:42.738 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=plugin.loader t=2024-09-16T10:34:42.430007638Z level=info msg="Plugin registered" pluginID=input 2024-09-16T10:34:42.738 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=plugin.loader t=2024-09-16T10:34:42.456834098Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-16T10:34:42.739 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=plugin.loader t=2024-09-16T10:34:42.456852422Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-16T10:34:42.739 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=secrets t=2024-09-16T10:34:42.456902952Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-16T10:34:42.739 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=query_data t=2024-09-16T10:34:42.458584903Z level=info msg="Query Service initialization" 2024-09-16T10:34:42.739 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:42 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=live.push_http t=2024-09-16T10:34:42.460848107Z level=info msg="Live Push Gateway initialization" 2024-09-16T10:34:43.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:43 smithi164 ceph-mon[40346]: Deploying daemon node-exporter.a on smithi164 2024-09-16T10:34:43.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:43 smithi164 ceph-mon[25828]: Deploying daemon node-exporter.a on smithi164 2024-09-16T10:34:43.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-mon[30341]: Deploying daemon node-exporter.a on smithi164 2024-09-16T10:34:43.485 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=infra.usagestats.collector t=2024-09-16T10:34:43.184902416Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-16T10:34:43.485 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=provisioning.datasources t=2024-09-16T10:34:43.240546566Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-16T10:34:43.485 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=provisioning.datasources t=2024-09-16T10:34:43.27260293Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-16T10:34:43.485 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=provisioning.alerting t=2024-09-16T10:34:43.294130386Z level=info msg="starting to provision alerting" 2024-09-16T10:34:43.486 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=provisioning.alerting t=2024-09-16T10:34:43.294156634Z level=info msg="finished to provision alerting" 2024-09-16T10:34:43.486 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=grafanaStorageLogger t=2024-09-16T10:34:43.294333032Z level=info msg="storage starting" 2024-09-16T10:34:43.486 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=ngalert.state.manager t=2024-09-16T10:34:43.294335576Z level=info msg="Warming state cache for startup" 2024-09-16T10:34:43.486 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=http.server t=2024-09-16T10:34:43.295974688Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-16T10:34:43.486 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=sqlstore.transactions t=2024-09-16T10:34:43.30510646Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T10:34:43.486 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=ngalert.state.manager t=2024-09-16T10:34:43.30517813Z level=info msg="State cache has been initialized" states=0 duration=10.841843ms 2024-09-16T10:34:43.486 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=ticker t=2024-09-16T10:34:43.305231525Z level=info msg=starting first_tick=2024-09-16T10:34:50Z 2024-09-16T10:34:43.486 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=ngalert.multiorg.alertmanager t=2024-09-16T10:34:43.305254015Z level=info msg="starting MultiOrg Alertmanager" 2024-09-16T10:34:43.486 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=sqlstore.transactions t=2024-09-16T10:34:43.359165028Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T10:34:43.486 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:34:43 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a[70935]: logger=sqlstore.transactions t=2024-09-16T10:34:43.369598644Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-16T10:34:44.120 INFO:teuthology.orchestra.run.smithi164.stdout:167 167 2024-09-16T10:34:44.157 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ::ffff:172.21.15.175 - - [16/Sep/2024:10:34:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T10:34:44.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:44 smithi164 ceph-mon[40346]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:44.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:44 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:44.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:44 smithi164 ceph-mon[25828]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:44.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:44 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:44.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:44 smithi175 ceph-mon[30341]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:44.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:44 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:44.768 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:44 smithi164 bash[84700]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-16T10:34:44.787 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T10:34:44.990 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:45.112 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:45 smithi164 bash[84700]: Getting image source signatures 2024-09-16T10:34:45.112 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:45 smithi164 bash[84700]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-16T10:34:45.112 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:45 smithi164 bash[84700]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-16T10:34:45.112 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:45 smithi164 bash[84700]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-16T10:34:46.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:46 smithi164 ceph-mon[40346]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:46.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:46 smithi164 ceph-mon[25828]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:46.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:46 smithi175 ceph-mon[30341]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:46.829 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:46 smithi164 bash[84700]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-16T10:34:46.829 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:46 smithi164 bash[84700]: Writing manifest to image destination 2024-09-16T10:34:47.469 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 podman[84700]: 2024-09-16 10:34:47.101443758 +0000 UTC m=+3.169438188 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-16T10:34:47.469 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 podman[84700]: 2024-09-16 10:34:47.350933733 +0000 UTC m=+3.418928205 container create 707b167eba288a4cfcff655dbf5f9a5c76d9bad49540f48c826bbe03b902866f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T10:34:47.831 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 podman[84700]: 2024-09-16 10:34:47.754867095 +0000 UTC m=+3.822861525 container init 707b167eba288a4cfcff655dbf5f9a5c76d9bad49540f48c826bbe03b902866f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T10:34:47.832 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 podman[84700]: 2024-09-16 10:34:47.75962087 +0000 UTC m=+3.827615302 container start 707b167eba288a4cfcff655dbf5f9a5c76d9bad49540f48c826bbe03b902866f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T10:34:47.832 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.763Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-16T10:34:47.832 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.763Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-16T10:34:47.832 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T10:34:47.832 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T10:34:47.832 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T10:34:47.832 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T10:34:47.832 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T10:34:47.832 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T10:34:47.832 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.764Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T10:34:47.833 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T10:34:47.834 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T10:34:47.835 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T10:34:47.835 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T10:34:47.835 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T10:34:47.835 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T10:34:47.835 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T10:34:47.835 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T10:34:47.835 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T10:34:47.835 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T10:34:47.836 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T10:34:47.836 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T10:34:47.836 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T10:34:47.836 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T10:34:47.836 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T10:34:47.836 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T10:34:47.836 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T10:34:47.836 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-16T10:34:47.836 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a[84953]: ts=2024-09-16T10:34:47.765Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T10:34:47.836 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 bash[84700]: 707b167eba288a4cfcff655dbf5f9a5c76d9bad49540f48c826bbe03b902866f 2024-09-16T10:34:48.082 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:34:47 smithi164 systemd[1]: Started Ceph node-exporter.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:34:48.261 INFO:teuthology.orchestra.run.smithi164.stdout:Backend: cephadm 2024-09-16T10:34:48.261 INFO:teuthology.orchestra.run.smithi164.stdout:Available: Yes 2024-09-16T10:34:48.261 INFO:teuthology.orchestra.run.smithi164.stdout:Paused: No 2024-09-16T10:34:48.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:48 smithi164 ceph-mon[25828]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:48.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:48 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:48.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:48 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:48.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:48 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:48.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:48 smithi164 ceph-mon[40346]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:48.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:48 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:48.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:48 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:48.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:48 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:48.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:48 smithi175 ceph-mon[30341]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:48.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:48 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:48.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:48 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:48.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:48 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:48.950 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:48 smithi175 systemd[1]: Starting Ceph node-exporter.b for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:34:49.009 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T10:34:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:49.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:49 smithi175 ceph-mon[30341]: Deploying daemon node-exporter.b on smithi175 2024-09-16T10:34:49.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2024-09-16T10:34:49.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:49 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2024-09-16T10:34:49.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:49 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:49 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:49 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:49 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:49.236 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:48 smithi175 bash[71204]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-16T10:34:49.329 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[25828]: Deploying daemon node-exporter.b on smithi175 2024-09-16T10:34:49.329 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2024-09-16T10:34:49.329 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.329 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2024-09-16T10:34:49.329 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.329 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.329 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.329 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.330 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.330 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:49.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[40346]: Deploying daemon node-exporter.b on smithi175 2024-09-16T10:34:49.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2024-09-16T10:34:49.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]: dispatch 2024-09-16T10:34:49.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.330 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-16T10:34:49.331 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:49 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:34:50.435 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[40346]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:50.435 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[40346]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:50.435 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]': finished 2024-09-16T10:34:50.435 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-16T10:34:50.435 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-16T10:34:50.435 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-16T10:34:50.435 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[40346]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T10:34:50.435 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[40346]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:50.436 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[25828]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:50.436 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[25828]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:50.436 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]': finished 2024-09-16T10:34:50.436 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-16T10:34:50.436 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-16T10:34:50.436 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-16T10:34:50.436 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[25828]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T10:34:50.436 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:50 smithi164 ceph-mon[25828]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:50.485 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:34:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug there is no tcmu-runner data available 2024-09-16T10:34:50.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:50 smithi175 ceph-mon[30341]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:50.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:50 smithi175 ceph-mon[30341]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:34:50.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:50 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [7, 2]}]': finished 2024-09-16T10:34:50.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:50 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-16T10:34:50.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:50 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-16T10:34:50.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:50 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-16T10:34:50.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:50 smithi175 ceph-mon[30341]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T10:34:50.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:50 smithi175 ceph-mon[30341]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:34:50.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:50 smithi175 bash[71204]: Getting image source signatures 2024-09-16T10:34:50.487 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:50 smithi175 bash[71204]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-16T10:34:50.487 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:50 smithi175 bash[71204]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-16T10:34:50.487 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:50 smithi175 bash[71204]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-16T10:34:51.109 INFO:teuthology.orchestra.run.smithi164.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T10:34:51.109 INFO:teuthology.orchestra.run.smithi164.stdout:alertmanager.a smithi164 *:9093,9094 starting - - - - 2024-09-16T10:34:51.109 INFO:teuthology.orchestra.run.smithi164.stdout:grafana.a smithi175 *:3000 starting - - - - 2024-09-16T10:34:51.109 INFO:teuthology.orchestra.run.smithi164.stdout:iscsi.iscsi.a smithi175 *:5000 running (85s) 58s ago 85s 56.6M - 3.8 67ddd33fa72f 1e9a78c1fad7 2024-09-16T10:34:51.109 INFO:teuthology.orchestra.run.smithi164.stdout:mgr.x smithi175 *:8443,8765 running (5m) 58s ago 5m 435M - 18.2.4-845-ge4e227b3 67ddd33fa72f e2b9b8042aca 2024-09-16T10:34:51.109 INFO:teuthology.orchestra.run.smithi164.stdout:mgr.y smithi164 *:9283,8765 running (6m) 57s ago 6m 465M - 18.2.4-845-ge4e227b3 67ddd33fa72f 3569545997fc 2024-09-16T10:34:51.109 INFO:teuthology.orchestra.run.smithi164.stdout:mon.a smithi164 running (6m) 57s ago 6m 50.5M 2048M 18.2.4-845-ge4e227b3 67ddd33fa72f 3513d0b825e1 2024-09-16T10:34:51.109 INFO:teuthology.orchestra.run.smithi164.stdout:mon.b smithi175 running (5m) 58s ago 5m 50.3M 2048M 18.2.4-845-ge4e227b3 67ddd33fa72f 325d79bb3a2e 2024-09-16T10:34:51.109 INFO:teuthology.orchestra.run.smithi164.stdout:mon.c smithi164 running (5m) 57s ago 5m 47.2M 2048M 18.2.4-845-ge4e227b3 67ddd33fa72f 35e6e621943d 2024-09-16T10:34:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:node-exporter.a smithi164 *:9100 starting - - - - 2024-09-16T10:34:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:osd.0 smithi164 running (4m) 57s ago 4m 53.9M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f 3a8d46469769 2024-09-16T10:34:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:osd.1 smithi164 running (4m) 57s ago 4m 47.0M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f ce0f5ec95fdd 2024-09-16T10:34:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:osd.2 smithi164 running (3m) 57s ago 3m 45.0M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f 07f0b8772cd0 2024-09-16T10:34:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:osd.3 smithi164 running (3m) 57s ago 3m 42.0M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f 04c78a10d66c 2024-09-16T10:34:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:osd.4 smithi175 running (3m) 58s ago 3m 42.3M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f 079392e29ec9 2024-09-16T10:34:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:osd.5 smithi175 running (2m) 58s ago 2m 46.3M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f 4484c1b38341 2024-09-16T10:34:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:osd.6 smithi175 running (2m) 58s ago 2m 50.8M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f db31cf7d699e 2024-09-16T10:34:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:osd.7 smithi175 running (119s) 58s ago 119s 54.5M 3760M 18.2.4-845-ge4e227b3 67ddd33fa72f cdfe990d2f05 2024-09-16T10:34:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:prometheus.a smithi175 *:9095 running (68s) 58s ago 68s 28.0M - 2.43.0 a07b618ecd1d 49b322f3700f 2024-09-16T10:34:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:rgw.foo.a smithi164 *:80 running (99s) 57s ago 98s 73.7M - 18.2.4-845-ge4e227b3 67ddd33fa72f 79aded71f0dc 2024-09-16T10:34:51.434 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:51 smithi164 ceph-mon[40346]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T10:34:51.434 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:51 smithi164 ceph-mon[40346]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:51.434 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:51 smithi164 ceph-mon[40346]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:51.434 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:51 smithi164 ceph-mon[25828]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T10:34:51.434 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:51 smithi164 ceph-mon[25828]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:51.434 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:51 smithi164 ceph-mon[25828]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:51.485 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:51 smithi175 bash[71204]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-16T10:34:51.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:51 smithi175 ceph-mon[30341]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T10:34:51.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:51 smithi175 ceph-mon[30341]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:34:51.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:51 smithi175 ceph-mon[30341]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:51.847 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T10:34:51.882 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:51 smithi175 bash[71204]: Writing manifest to image destination 2024-09-16T10:34:52.045 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:52.145 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:51 smithi175 podman[71204]: 2024-09-16 10:34:51.801446354 +0000 UTC m=+2.945281445 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-16T10:34:52.145 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:51 smithi175 podman[71204]: 2024-09-16 10:34:51.900078343 +0000 UTC m=+3.043913430 container create 3e319ee4b38bfa7928c662e056580c05e87c3f208d4b4dfb0bee0c82015232bc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T10:34:52.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:52 smithi164 ceph-mon[25828]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-09-16T10:34:52.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:52 smithi164 ceph-mon[40346]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-09-16T10:34:52.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-mon[30341]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-09-16T10:34:52.485 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 podman[71204]: 2024-09-16 10:34:52.144157204 +0000 UTC m=+3.287992291 container init 3e319ee4b38bfa7928c662e056580c05e87c3f208d4b4dfb0bee0c82015232bc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T10:34:52.485 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 podman[71204]: 2024-09-16 10:34:52.149227757 +0000 UTC m=+3.293062846 container start 3e319ee4b38bfa7928c662e056580c05e87c3f208d4b4dfb0bee0c82015232bc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T10:34:52.485 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.152Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-16T10:34:52.485 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.152Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T10:34:52.486 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T10:34:52.487 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T10:34:52.487 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T10:34:52.487 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T10:34:52.487 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.153Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T10:34:52.487 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T10:34:52.487 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T10:34:52.487 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T10:34:52.488 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T10:34:52.489 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T10:34:52.489 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T10:34:52.489 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T10:34:52.489 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T10:34:52.489 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T10:34:52.489 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T10:34:52.489 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T10:34:52.489 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T10:34:52.489 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T10:34:52.489 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T10:34:52.489 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T10:34:52.490 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T10:34:52.490 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T10:34:52.490 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T10:34:52.490 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T10:34:52.490 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T10:34:52.490 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T10:34:52.490 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T10:34:52.490 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-16T10:34:52.490 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b[71281]: ts=2024-09-16T10:34:52.154Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T10:34:52.490 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 bash[71204]: 3e319ee4b38bfa7928c662e056580c05e87c3f208d4b4dfb0bee0c82015232bc 2024-09-16T10:34:52.490 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:34:52 smithi175 systemd[1]: Started Ceph node-exporter.b for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:34:53.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:34:53.438 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[40346]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T10:34:53.438 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.439 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.439 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.439 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.439 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.439 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:34:53.440 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:53 smithi164 ceph-mon[25828]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T10:34:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:53 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:53 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T10:34:53.736 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:53 smithi175 ceph-mon[30341]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-16T10:34:54.079 INFO:teuthology.orchestra.run.smithi164.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T10:34:54.079 INFO:teuthology.orchestra.run.smithi164.stdout:alertmanager ?:9093,9094 0/1 - 82s smithi164=a;count:1 2024-09-16T10:34:54.079 INFO:teuthology.orchestra.run.smithi164.stdout:grafana ?:3000 0/1 - 77s smithi175=a;count:1 2024-09-16T10:34:54.079 INFO:teuthology.orchestra.run.smithi164.stdout:iscsi.datapool ?:5000 1/1 61s ago 93s smithi175=iscsi.a;count:1 2024-09-16T10:34:54.079 INFO:teuthology.orchestra.run.smithi164.stdout:mgr 2/2 61s ago 5m smithi164=y;smithi175=x;count:2 2024-09-16T10:34:54.080 INFO:teuthology.orchestra.run.smithi164.stdout:mon 3/3 61s ago 5m smithi164:172.21.15.164=a;smithi164:[v2:172.21.15.164:3301,v1:172.21.15.164:6790]=c;smithi175:172.21.15.175=b;count:3 2024-09-16T10:34:54.080 INFO:teuthology.orchestra.run.smithi164.stdout:node-exporter ?:9100 0/2 - 85s smithi164=a;smithi175=b;count:2 2024-09-16T10:34:54.080 INFO:teuthology.orchestra.run.smithi164.stdout:osd 8 61s ago - 2024-09-16T10:34:54.080 INFO:teuthology.orchestra.run.smithi164.stdout:prometheus ?:9095 1/1 61s ago 89s smithi175=a;count:1 2024-09-16T10:34:54.080 INFO:teuthology.orchestra.run.smithi164.stdout:rgw.foo.a ?:80 1/1 60s ago 101s smithi164=foo.a;count:1 2024-09-16T10:34:54.443 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:34:54 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ::ffff:172.21.15.175 - - [16/Sep/2024:10:34:54] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T10:34:55.045 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T10:34:55.270 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:34:55.391 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:55 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:55.392 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:55 smithi164 ceph-mon[40346]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:55.392 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:55 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:55.392 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:55 smithi164 ceph-mon[25828]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:55.467 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:55 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:55.467 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:55 smithi175 ceph-mon[30341]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:56.355 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:56 smithi175 ceph-mon[30341]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-16T10:34:56.355 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:56 smithi175 ceph-mon[30341]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-09-16T10:34:56.355 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:56 smithi175 ceph-mon[30341]: Cluster is now healthy 2024-09-16T10:34:56.379 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:56 smithi164 ceph-mon[40346]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-16T10:34:56.379 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:56 smithi164 ceph-mon[40346]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-09-16T10:34:56.380 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:56 smithi164 ceph-mon[40346]: Cluster is now healthy 2024-09-16T10:34:56.380 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:56 smithi164 ceph-mon[25828]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-16T10:34:56.380 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:56 smithi164 ceph-mon[25828]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-09-16T10:34:56.380 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:56 smithi164 ceph-mon[25828]: Cluster is now healthy 2024-09-16T10:34:57.659 INFO:teuthology.orchestra.run.smithi164.stdout:HOST ADDR LABELS STATUS 2024-09-16T10:34:57.660 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164 172.21.15.164 2024-09-16T10:34:57.660 INFO:teuthology.orchestra.run.smithi164.stdout:smithi175 172.21.15.175 2024-09-16T10:34:57.660 INFO:teuthology.orchestra.run.smithi164.stdout:2 hosts in cluster 2024-09-16T10:34:57.921 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:57 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:57.922 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:57 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:57.922 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:57 smithi164 ceph-mon[40346]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-16T10:34:57.922 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:57 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:57.922 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:57 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:57.922 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:57 smithi164 ceph-mon[25828]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-16T10:34:57.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:57 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:57.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:57 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:34:57.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:57 smithi175 ceph-mon[30341]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-16T10:34:58.321 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T10:34:58.943 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:34:58 smithi164 ceph-mon[40346]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:58.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:34:58 smithi164 ceph-mon[25828]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:58.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:34:58 smithi175 ceph-mon[30341]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:34:59.072 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:35:00.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[40346]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-09-16T10:35:00.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:00.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:00.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:35:00.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:35:00.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:00.176 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[25828]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-09-16T10:35:00.177 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:00.177 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:00.177 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:35:00.177 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:35:00.177 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:00 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:00.485 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:35:00 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug there is no tcmu-runner data available 2024-09-16T10:35:00.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:00 smithi175 ceph-mon[30341]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-09-16T10:35:00.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:00 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:00 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:00 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T10:35:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:00 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T10:35:00.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:00 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:01.409 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:01 smithi164 ceph-mon[40346]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T10:35:01.409 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:01 smithi164 ceph-mon[40346]: Reconfiguring daemon alertmanager.a on smithi164 2024-09-16T10:35:01.409 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:01 smithi164 ceph-mon[40346]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:35:01.409 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:01 smithi164 ceph-mon[25828]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T10:35:01.409 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:01 smithi164 ceph-mon[25828]: Reconfiguring daemon alertmanager.a on smithi164 2024-09-16T10:35:01.410 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:01 smithi164 ceph-mon[25828]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:35:01.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:01 smithi175 ceph-mon[30341]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T10:35:01.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:01 smithi175 ceph-mon[30341]: Reconfiguring daemon alertmanager.a on smithi164 2024-09-16T10:35:01.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:01 smithi175 ceph-mon[30341]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:35:02.179 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:02 smithi164 ceph-mon[40346]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 944 B/s rd, 0 op/s; 10 B/s, 1 objects/s recovering 2024-09-16T10:35:02.180 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:02 smithi164 ceph-mon[25828]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 944 B/s rd, 0 op/s; 10 B/s, 1 objects/s recovering 2024-09-16T10:35:02.340 INFO:teuthology.orchestra.run.smithi164.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T10:35:02.340 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300H0400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T10:35:02.340 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164 /dev/nvme1n1 ssd Linux_3cbbed3bfe7af15735a8 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T10:35:02.340 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164 /dev/nvme2n1 ssd Linux_e1380edeeeaf39d69a5f 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T10:35:02.340 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164 /dev/nvme3n1 ssd Linux_f88aa3bd38a9077199db 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T10:35:02.340 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164 /dev/nvme4n1 ssd Linux_8aa0d2e26cbdb8078d05 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T10:35:02.340 INFO:teuthology.orchestra.run.smithi164.stdout:smithi175 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300M0400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-16T10:35:02.341 INFO:teuthology.orchestra.run.smithi164.stdout:smithi175 /dev/nvme1n1 ssd Linux_63e52e71259bdbfe9151 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T10:35:02.341 INFO:teuthology.orchestra.run.smithi164.stdout:smithi175 /dev/nvme2n1 ssd Linux_e7977759e4910d9f0efe 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T10:35:02.341 INFO:teuthology.orchestra.run.smithi164.stdout:smithi175 /dev/nvme3n1 ssd Linux_6e5cbfe78f644349609f 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T10:35:02.341 INFO:teuthology.orchestra.run.smithi164.stdout:smithi175 /dev/nvme4n1 ssd Linux_67281d28ca84bcde9cfa 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-16T10:35:02.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:02 smithi175 ceph-mon[30341]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 944 B/s rd, 0 op/s; 10 B/s, 1 objects/s recovering 2024-09-16T10:35:03.768 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls --format yaml' 2024-09-16T10:35:04.375 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:04 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: ::ffff:172.21.15.175 - - [16/Sep/2024:10:35:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-16T10:35:04.376 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:04 smithi164 ceph-mon[25828]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:35:04.377 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:04 smithi164 ceph-mon[25828]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T10:35:04.377 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:04 smithi164 ceph-mon[40346]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:35:04.377 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:04 smithi164 ceph-mon[40346]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T10:35:04.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:04 smithi175 ceph-mon[30341]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:35:04.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:04 smithi175 ceph-mon[30341]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T10:35:04.505 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:35:04.626 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:04 smithi164 systemd[1]: Stopping Ceph alertmanager.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:05.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:05 smithi164 ceph-mon[40346]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:35:05.193 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:04 smithi164 podman[86765]: 2024-09-16 10:35:04.97781776 +0000 UTC m=+0.524961840 container died 5805a74a8c8af405cad6ed6d207a7951fe43bc669d0f533428bc8dd0adaf9c6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=funny_booth, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-16T10:35:05.194 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:05 smithi164 ceph-mon[25828]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:35:05.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:05 smithi175 ceph-mon[30341]: from='mgr.14556 172.21.15.164:0/548521750' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T10:35:06.193 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:06 smithi164 ceph-mon[25828]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T10:35:06.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:06 smithi164 ceph-mon[40346]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T10:35:06.194 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:05 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[83411]: ts=2024-09-16T10:35:05.956Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-16T10:35:06.194 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:05 smithi164 podman[86863]: 2024-09-16 10:35:05.976847283 +0000 UTC m=+0.754872723 container died d1a5e1bf4e157459e805f3182edb6df132ecf12f77a1b7306f0dce562d27c061 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:06.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:06 smithi175 ceph-mon[30341]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 9 B/s, 0 objects/s recovering 2024-09-16T10:35:06.661 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:06 smithi164 podman[86863]: 2024-09-16 10:35:06.421468399 +0000 UTC m=+1.199493828 container cleanup d1a5e1bf4e157459e805f3182edb6df132ecf12f77a1b7306f0dce562d27c061 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:06.661 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:06 smithi164 bash[86863]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a 2024-09-16T10:35:07.069 INFO:teuthology.orchestra.run.smithi164.stdout:service_type: alertmanager 2024-09-16T10:35:07.069 INFO:teuthology.orchestra.run.smithi164.stdout:service_name: alertmanager 2024-09-16T10:35:07.069 INFO:teuthology.orchestra.run.smithi164.stdout:placement: 2024-09-16T10:35:07.070 INFO:teuthology.orchestra.run.smithi164.stdout: count: 1 2024-09-16T10:35:07.070 INFO:teuthology.orchestra.run.smithi164.stdout: hosts: 2024-09-16T10:35:07.070 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi164=a 2024-09-16T10:35:07.070 INFO:teuthology.orchestra.run.smithi164.stdout:status: 2024-09-16T10:35:07.070 INFO:teuthology.orchestra.run.smithi164.stdout: created: '2024-09-16T10:33:32.030087Z' 2024-09-16T10:35:07.070 INFO:teuthology.orchestra.run.smithi164.stdout: last_refresh: '2024-09-16T10:34:59.048344Z' 2024-09-16T10:35:07.070 INFO:teuthology.orchestra.run.smithi164.stdout: ports: 2024-09-16T10:35:07.070 INFO:teuthology.orchestra.run.smithi164.stdout: - 9093 2024-09-16T10:35:07.070 INFO:teuthology.orchestra.run.smithi164.stdout: - 9094 2024-09-16T10:35:07.070 INFO:teuthology.orchestra.run.smithi164.stdout: running: 1 2024-09-16T10:35:07.071 INFO:teuthology.orchestra.run.smithi164.stdout: size: 1 2024-09-16T10:35:07.071 INFO:teuthology.orchestra.run.smithi164.stdout:events: 2024-09-16T10:35:07.071 INFO:teuthology.orchestra.run.smithi164.stdout:- 2024-09-16T10:34:30.571261Z service:alertmanager [INFO] "service was created" 2024-09-16T10:35:07.071 INFO:teuthology.orchestra.run.smithi164.stdout:--- 2024-09-16T10:35:07.071 INFO:teuthology.orchestra.run.smithi164.stdout:service_type: grafana 2024-09-16T10:35:07.071 INFO:teuthology.orchestra.run.smithi164.stdout:service_name: grafana 2024-09-16T10:35:07.071 INFO:teuthology.orchestra.run.smithi164.stdout:placement: 2024-09-16T10:35:07.071 INFO:teuthology.orchestra.run.smithi164.stdout: count: 1 2024-09-16T10:35:07.071 INFO:teuthology.orchestra.run.smithi164.stdout: hosts: 2024-09-16T10:35:07.071 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi175=a 2024-09-16T10:35:07.071 INFO:teuthology.orchestra.run.smithi164.stdout:spec: 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout: anonymous_access: true 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout: protocol: https 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout:status: 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout: created: '2024-09-16T10:33:36.419034Z' 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout: last_refresh: '2024-09-16T10:34:56.651404Z' 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout: ports: 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout: - 3000 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout: running: 1 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout: size: 1 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout:events: 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout:- 2024-09-16T10:34:41.718940Z service:grafana [INFO] "service was created" 2024-09-16T10:35:07.072 INFO:teuthology.orchestra.run.smithi164.stdout:--- 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout:service_type: iscsi 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout:service_id: datapool 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout:service_name: iscsi.datapool 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout:placement: 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout: count: 1 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout: hosts: 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi175=iscsi.a 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout:spec: 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout: api_password: admin 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout: api_port: 5000 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout: api_user: admin 2024-09-16T10:35:07.073 INFO:teuthology.orchestra.run.smithi164.stdout: pool: datapool 2024-09-16T10:35:07.074 INFO:teuthology.orchestra.run.smithi164.stdout: trusted_ip_list: 172.21.15.175 2024-09-16T10:35:07.074 INFO:teuthology.orchestra.run.smithi164.stdout:status: 2024-09-16T10:35:07.074 INFO:teuthology.orchestra.run.smithi164.stdout: created: '2024-09-16T10:33:20.369579Z' 2024-09-16T10:35:07.074 INFO:teuthology.orchestra.run.smithi164.stdout: last_refresh: '2024-09-16T10:34:56.651294Z' 2024-09-16T10:35:07.074 INFO:teuthology.orchestra.run.smithi164.stdout: ports: 2024-09-16T10:35:07.074 INFO:teuthology.orchestra.run.smithi164.stdout: - 5000 2024-09-16T10:35:07.074 INFO:teuthology.orchestra.run.smithi164.stdout: running: 1 2024-09-16T10:35:07.074 INFO:teuthology.orchestra.run.smithi164.stdout: size: 1 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout:--- 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout:service_type: mgr 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout:service_name: mgr 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout:placement: 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout: count: 2 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout: hosts: 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi164=y 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi175=x 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout:status: 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout: created: '2024-09-16T10:29:43.765809Z' 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout: last_refresh: '2024-09-16T10:34:56.651008Z' 2024-09-16T10:35:07.075 INFO:teuthology.orchestra.run.smithi164.stdout: running: 2 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout: size: 2 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout:--- 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout:service_type: mon 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout:service_name: mon 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout:placement: 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout: count: 3 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout: hosts: 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi164:172.21.15.164=a 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi164:[v2:172.21.15.164:3301,v1:172.21.15.164:6790]=c 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi175:172.21.15.175=b 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout:status: 2024-09-16T10:35:07.076 INFO:teuthology.orchestra.run.smithi164.stdout: created: '2024-09-16T10:29:16.437490Z' 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout: last_refresh: '2024-09-16T10:34:56.650913Z' 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout: running: 3 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout: size: 3 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout:--- 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout:service_type: node-exporter 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout:service_name: node-exporter 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout:placement: 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout: count: 2 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout: hosts: 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi164=a 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi175=b 2024-09-16T10:35:07.077 INFO:teuthology.orchestra.run.smithi164.stdout:status: 2024-09-16T10:35:07.078 INFO:teuthology.orchestra.run.smithi164.stdout: created: '2024-09-16T10:33:28.221046Z' 2024-09-16T10:35:07.078 INFO:teuthology.orchestra.run.smithi164.stdout: last_refresh: '2024-09-16T10:34:56.651459Z' 2024-09-16T10:35:07.078 INFO:teuthology.orchestra.run.smithi164.stdout: ports: 2024-09-16T10:35:07.078 INFO:teuthology.orchestra.run.smithi164.stdout: - 9100 2024-09-16T10:35:07.078 INFO:teuthology.orchestra.run.smithi164.stdout: running: 2 2024-09-16T10:35:07.078 INFO:teuthology.orchestra.run.smithi164.stdout: size: 2 2024-09-16T10:35:07.078 INFO:teuthology.orchestra.run.smithi164.stdout:events: 2024-09-16T10:35:07.078 INFO:teuthology.orchestra.run.smithi164.stdout:- 2024-09-16T10:34:52.319728Z service:node-exporter [INFO] "service was created" 2024-09-16T10:35:07.078 INFO:teuthology.orchestra.run.smithi164.stdout:--- 2024-09-16T10:35:07.078 INFO:teuthology.orchestra.run.smithi164.stdout:service_type: osd 2024-09-16T10:35:07.078 INFO:teuthology.orchestra.run.smithi164.stdout:service_name: osd 2024-09-16T10:35:07.079 INFO:teuthology.orchestra.run.smithi164.stdout:unmanaged: true 2024-09-16T10:35:07.079 INFO:teuthology.orchestra.run.smithi164.stdout:spec: 2024-09-16T10:35:07.079 INFO:teuthology.orchestra.run.smithi164.stdout: filter_logic: AND 2024-09-16T10:35:07.079 INFO:teuthology.orchestra.run.smithi164.stdout: objectstore: bluestore 2024-09-16T10:35:07.079 INFO:teuthology.orchestra.run.smithi164.stdout:status: 2024-09-16T10:35:07.079 INFO:teuthology.orchestra.run.smithi164.stdout: container_image_id: 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 2024-09-16T10:35:07.079 INFO:teuthology.orchestra.run.smithi164.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T10:35:07.079 INFO:teuthology.orchestra.run.smithi164.stdout: last_refresh: '2024-09-16T10:34:56.651068Z' 2024-09-16T10:35:07.079 INFO:teuthology.orchestra.run.smithi164.stdout: running: 8 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout: size: 8 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout:--- 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout:service_type: prometheus 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout:service_name: prometheus 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout:placement: 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout: count: 1 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout: hosts: 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi175=a 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout:status: 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout: created: '2024-09-16T10:33:24.731687Z' 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout: last_refresh: '2024-09-16T10:34:56.651349Z' 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout: ports: 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout: - 9095 2024-09-16T10:35:07.080 INFO:teuthology.orchestra.run.smithi164.stdout: running: 1 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout: size: 1 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout:events: 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout:- 2024-09-16T10:34:52.326451Z service:prometheus [INFO] "service was created" 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout:--- 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout:service_type: rgw 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout:service_id: foo.a 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout:service_name: rgw.foo.a 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout:placement: 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout: count: 1 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout: hosts: 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout: - smithi164=foo.a 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout:status: 2024-09-16T10:35:07.081 INFO:teuthology.orchestra.run.smithi164.stdout: created: '2024-09-16T10:33:12.678192Z' 2024-09-16T10:35:07.082 INFO:teuthology.orchestra.run.smithi164.stdout: last_refresh: '2024-09-16T10:34:59.048287Z' 2024-09-16T10:35:07.082 INFO:teuthology.orchestra.run.smithi164.stdout: ports: 2024-09-16T10:35:07.082 INFO:teuthology.orchestra.run.smithi164.stdout: - 80 2024-09-16T10:35:07.082 INFO:teuthology.orchestra.run.smithi164.stdout: running: 1 2024-09-16T10:35:07.082 INFO:teuthology.orchestra.run.smithi164.stdout: size: 1 2024-09-16T10:35:07.443 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:07 smithi164 podman[86905]: 2024-09-16 10:35:07.127528025 +0000 UTC m=+1.152508710 container remove d1a5e1bf4e157459e805f3182edb6df132ecf12f77a1b7306f0dce562d27c061 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:07.443 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:07 smithi164 podman[86905]: 2024-09-16 10:35:07.211346084 +0000 UTC m=+1.236326769 volume remove feb913328b4f0d23752e3e65e956bbe969cb0da8ee2a16d026390ff6ce95f8e9 2024-09-16T10:35:08.006 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-09-16T10:35:08.284 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-mon[40346]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:35:08.284 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-mon[40346]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-16T10:35:08.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-mon[25828]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:35:08.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-mon[25828]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-16T10:35:08.285 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-16T10:35:08.285 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 systemd[1]: Stopped Ceph alertmanager.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:35:08.285 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 systemd[1]: Starting Ceph alertmanager.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:08.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:08 smithi175 ceph-mon[30341]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:35:08.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:08 smithi175 ceph-mon[30341]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-16T10:35:08.579 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 podman[87103]: 2024-09-16 10:35:08.185677848 +0000 UTC m=+0.019492988 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-16T10:35:08.580 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 podman[87103]: 2024-09-16 10:35:08.283332076 +0000 UTC m=+0.117147208 volume create 7fb02c3c65b62ea9ad02c214a056456a7f9aea46fb24851dd0d3ddde1004d254 2024-09-16T10:35:08.580 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 podman[87103]: 2024-09-16 10:35:08.392245562 +0000 UTC m=+0.226060686 container create 762aba524889ef76181c2316094d44a2f7668794211967c70d5accd3a5f8491b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:08.853 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T10:35:08.943 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 podman[87103]: 2024-09-16 10:35:08.678876337 +0000 UTC m=+0.512691463 container init 762aba524889ef76181c2316094d44a2f7668794211967c70d5accd3a5f8491b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:08.943 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 podman[87103]: 2024-09-16 10:35:08.683530834 +0000 UTC m=+0.517345960 container start 762aba524889ef76181c2316094d44a2f7668794211967c70d5accd3a5f8491b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:08.943 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:35:08.695Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-16T10:35:08.944 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:35:08.695Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-16T10:35:08.944 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:35:08.696Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.164 port=9094 2024-09-16T10:35:08.944 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:35:08.697Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-16T10:35:08.944 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:35:08.720Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T10:35:08.944 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:35:08.720Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T10:35:08.944 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:35:08.723Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-16T10:35:08.944 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:35:08.723Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-16T10:35:08.944 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 bash[87103]: 762aba524889ef76181c2316094d44a2f7668794211967c70d5accd3a5f8491b 2024-09-16T10:35:08.944 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:08 smithi164 systemd[1]: Started Ceph alertmanager.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:35:09.943 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:09 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:09.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:09 smithi164 ceph-mon[25828]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:09.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:09 smithi164 ceph-mon[25828]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T10:35:09.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:09 smithi164 ceph-mon[25828]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:35:09.944 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:09 smithi164 ceph-mon[25828]: Reconfiguring daemon prometheus.a on smithi175 2024-09-16T10:35:09.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:09 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:09.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:09 smithi164 ceph-mon[40346]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:09.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:09 smithi164 ceph-mon[40346]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T10:35:09.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:09 smithi164 ceph-mon[40346]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:35:09.944 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:09 smithi164 ceph-mon[40346]: Reconfiguring daemon prometheus.a on smithi175 2024-09-16T10:35:10.162 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:09 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:10.162 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:09 smithi175 ceph-mon[30341]: from='mgr.14556 ' entity='mgr.y' 2024-09-16T10:35:10.163 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:09 smithi175 ceph-mon[30341]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T10:35:10.163 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:09 smithi175 ceph-mon[30341]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T10:35:10.163 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:09 smithi175 ceph-mon[30341]: Reconfiguring daemon prometheus.a on smithi175 2024-09-16T10:35:10.163 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:35:10 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug there is no tcmu-runner data available 2024-09-16T10:35:10.718 INFO:teuthology.orchestra.run.smithi164.stdout:osd 8 14s ago - 2024-09-16T10:35:11.077 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:10 smithi164 ceph-mon[40346]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:35:11.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:10 smithi164 ceph-mon[25828]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:35:11.078 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:10 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:35:10.697Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000469615s 2024-09-16T10:35:11.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:10 smithi175 ceph-mon[30341]: from='client.14511 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T10:35:11.426 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T10:35:11.435 INFO:tasks.cephadm:Teardown begin 2024-09-16T10:35:11.435 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T10:35:11.465 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T10:35:11.495 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T10:35:11.496 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T10:35:11.521 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T10:35:11.552 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T10:35:11.553 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T10:35:11.553 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.a 2024-09-16T10:35:11.693 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:11 smithi164 systemd[1]: Stopping Ceph mon.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:12.174 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:11 smithi164 ceph-mon[40346]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:35:12.174 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:11 smithi164 ceph-mon[40346]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:35:12.175 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:11 smithi164 ceph-mon[25828]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:35:12.175 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:11 smithi164 ceph-mon[25828]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:35:12.175 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:11 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a[25805]: 2024-09-16T10:35:11.857+0000 7fb272c61640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:35:12.175 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:11 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a[25805]: 2024-09-16T10:35:11.857+0000 7fb272c61640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T10:35:12.175 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:11 smithi164 podman[87408]: 2024-09-16 10:35:11.888360078 +0000 UTC m=+0.137095857 container died 3513d0b825e1dbf0ee0759252d2a89ccf2129e55fd8088c2cf0fc5b7981a942c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b) 2024-09-16T10:35:12.230 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:11 smithi175 ceph-mon[30341]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T10:35:12.230 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:11 smithi175 ceph-mon[30341]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T10:35:12.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:12 smithi164 podman[87408]: 2024-09-16 10:35:12.388909234 +0000 UTC m=+0.637645007 container cleanup 3513d0b825e1dbf0ee0759252d2a89ccf2129e55fd8088c2cf0fc5b7981a942c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b) 2024-09-16T10:35:12.443 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 16 10:35:12 smithi164 bash[87408]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-a 2024-09-16T10:35:12.806 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T10:35:12.849 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:35:12.849 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T10:35:12.850 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-16T10:35:12.850 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.c 2024-09-16T10:35:13.164 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:12 smithi164 systemd[1]: Stopping Ceph mon.c for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:13.443 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:13 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:35:13] ENGINE Bus STOPPING 2024-09-16T10:35:13.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:13 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-c[40323]: 2024-09-16T10:35:13.161+0000 7f5e78117640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:35:13.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:13 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-c[40323]: 2024-09-16T10:35:13.161+0000 7f5e78117640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T10:35:13.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:13 smithi164 podman[87539]: 2024-09-16 10:35:13.213328304 +0000 UTC m=+0.145448483 container died 35e6e621943d6ca51c68140c0ec6923d0dd32de14dbe956cae468679efa8d471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T10:35:13.708 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:13 smithi164 podman[87539]: 2024-09-16 10:35:13.627185202 +0000 UTC m=+0.559305385 container cleanup 35e6e621943d6ca51c68140c0ec6923d0dd32de14dbe956cae468679efa8d471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-c, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T10:35:13.708 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:13 smithi164 bash[87539]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-c 2024-09-16T10:35:13.708 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:13 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:35:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T10:35:13.708 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:13 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:35:13] ENGINE Bus STOPPED 2024-09-16T10:35:13.708 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:13 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:35:13] ENGINE Bus STARTING 2024-09-16T10:35:13.708 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:13 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:35:13] ENGINE Serving on http://:::9283 2024-09-16T10:35:13.709 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:13 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: [16/Sep/2024:10:35:13] ENGINE Bus STARTED 2024-09-16T10:35:13.985 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:13 smithi175 systemd[1]: Stopping Ceph prometheus.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:14.083 INFO:journalctl@ceph.mon.c.smithi164.stdout:Sep 16 10:35:13 smithi164 podman[87556]: 2024-09-16 10:35:13.853893931 +0000 UTC m=+0.639845493 container remove 35e6e621943d6ca51c68140c0ec6923d0dd32de14dbe956cae468679efa8d471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-c, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T10:35:14.099 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T10:35:14.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:35:14.135 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-16T10:35:14.135 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-16T10:35:14.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.b 2024-09-16T10:35:14.341 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:14 smithi175 systemd[1]: Stopping Ceph mon.b for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:14.341 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T10:35:14.341 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-16T10:35:14.341 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-16T10:35:14.342 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T10:35:14.342 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-16T10:35:14.342 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-16T10:35:14.342 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T10:35:14.342 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-16T10:35:14.342 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-16T10:35:14.342 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T10:35:14.342 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-16T10:35:14.342 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[68894]: ts=2024-09-16T10:35:14.074Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-16T10:35:14.342 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 podman[72349]: 2024-09-16 10:35:14.084664121 +0000 UTC m=+0.132648083 container died 49b322f3700fdc48b9d6c95a058102ab28c2dfc7e00fdcdbc631d6dc4a7fbb4d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:14.631 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 podman[72349]: 2024-09-16 10:35:14.438866292 +0000 UTC m=+0.486850254 container cleanup 49b322f3700fdc48b9d6c95a058102ab28c2dfc7e00fdcdbc631d6dc4a7fbb4d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:14.631 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 bash[72349]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T10:35:14.632 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-b[30300]: 2024-09-16T10:35:14.576+0000 7f21dd42c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:35:14.632 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-b[30300]: 2024-09-16T10:35:14.576+0000 7f21dd42c640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T10:35:14.985 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:14 smithi175 podman[72444]: 2024-09-16 10:35:14.877812523 +0000 UTC m=+0.393584308 container died 325d79bb3a2ed5de9d7fd1d103b06ad3498181e65f3d22d89c2c64fe0bc164f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-b, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T10:35:14.985 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 bash[72478]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T10:35:14.985 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 podman[72366]: 2024-09-16 10:35:14.832466975 +0000 UTC m=+0.749652934 container remove 49b322f3700fdc48b9d6c95a058102ab28c2dfc7e00fdcdbc631d6dc4a7fbb4d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:14.986 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-16T10:35:14.986 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 systemd[1]: Stopped Ceph prometheus.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:35:14.986 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.125s CPU time. 2024-09-16T10:35:14.986 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:14 smithi175 systemd[1]: Starting Ceph prometheus.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:15.462 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:15 smithi175 podman[72444]: 2024-09-16 10:35:15.267993963 +0000 UTC m=+0.783765737 container cleanup 325d79bb3a2ed5de9d7fd1d103b06ad3498181e65f3d22d89c2c64fe0bc164f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T10:35:15.462 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:15 smithi175 bash[72444]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-b 2024-09-16T10:35:15.735 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 16 10:35:15 smithi175 podman[72489]: 2024-09-16 10:35:15.460887844 +0000 UTC m=+0.811952737 container remove 325d79bb3a2ed5de9d7fd1d103b06ad3498181e65f3d22d89c2c64fe0bc164f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=reef-e4e227b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T10:35:15.735 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 podman[72573]: 2024-09-16 10:35:15.585227685 +0000 UTC m=+0.019711495 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-16T10:35:15.735 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 podman[72573]: 2024-09-16 10:35:15.703425471 +0000 UTC m=+0.137909253 container create 4175c42b42c5064f10a4ebc7f65a32317f98c51c5f02c96e3a76a1092a84e620 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:16.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T10:35:16.143 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:35:16.144 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-16T10:35:16.144 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T10:35:16.144 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mgr.y 2024-09-16T10:35:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 podman[72573]: 2024-09-16 10:35:15.96381733 +0000 UTC m=+0.398301118 container init 4175c42b42c5064f10a4ebc7f65a32317f98c51c5f02c96e3a76a1092a84e620 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 podman[72573]: 2024-09-16 10:35:15.968834915 +0000 UTC m=+0.403318696 container start 4175c42b42c5064f10a4ebc7f65a32317f98c51c5f02c96e3a76a1092a84e620 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T10:35:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.992Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-16T10:35:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.992Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-16T10:35:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.992Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi175 (none))" 2024-09-16T10:35:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.992Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-16T10:35:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.992Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-16T10:35:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.994Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-16T10:35:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.994Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-16T10:35:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.995Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-16T10:35:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.995Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.997Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.997Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.72µs 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.997Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.998Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.998Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.999Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=40.88µs wal_replay_duration=1.616645ms wbl_replay_duration=117ns total_replay_duration=1.946315ms 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.999Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.999Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:15.999Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:16.020Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.630457ms db_storage=714ns remote_storage=991ns web_handler=329ns query_engine=499ns scrape=395.09µs scrape_sd=97.058µs notify=10.024µs notify_sd=8.01µs rules=19.815448ms tracing=6.088µs 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:16.020Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:35:16.020Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T10:35:16.237 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:16 smithi175 bash[72573]: 4175c42b42c5064f10a4ebc7f65a32317f98c51c5f02c96e3a76a1092a84e620 2024-09-16T10:35:16.238 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:35:16 smithi175 systemd[1]: Started Ceph prometheus.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:35:16.443 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:16 smithi164 systemd[1]: Stopping Ceph mgr.y for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:16.693 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:16 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:35:16.441+0000 7fe0a87d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:35:16.693 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:16 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y[26078]: 2024-09-16T10:35:16.441+0000 7fe0a87d9640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-16T10:35:16.693 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:16 smithi164 podman[87694]: 2024-09-16 10:35:16.472924706 +0000 UTC m=+0.139459554 container died 3569545997fca00d2f0c2c898dc258cb14b4b796b506dd16e43d7ddda0159296 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y, ceph=True, GIT_CLEAN=True, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T10:35:17.193 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:16 smithi164 podman[87694]: 2024-09-16 10:35:16.914826446 +0000 UTC m=+0.581361293 container cleanup 3569545997fca00d2f0c2c898dc258cb14b4b796b506dd16e43d7ddda0159296 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e4e227b37858f5109138494e01e5ba1ba7b8ffe8, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-16T10:35:17.193 INFO:journalctl@ceph.mgr.y.smithi164.stdout:Sep 16 10:35:16 smithi164 bash[87694]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-y 2024-09-16T10:35:17.332 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T10:35:17.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:35:17.369 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T10:35:17.369 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T10:35:17.369 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mgr.x 2024-09-16T10:35:17.682 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:35:17 smithi175 systemd[1]: Stopping Ceph mgr.x for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:17.962 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:35:17 smithi175 podman[72704]: 2024-09-16 10:35:17.681809682 +0000 UTC m=+0.117945635 container died e2b9b8042acade58c71c4dab9338d8576da94ce67f14892475db1bc798b8c51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T10:35:18.235 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:35:18 smithi175 podman[72704]: 2024-09-16 10:35:18.044128232 +0000 UTC m=+0.480264186 container cleanup e2b9b8042acade58c71c4dab9338d8576da94ce67f14892475db1bc798b8c51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T10:35:18.235 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 16 10:35:18 smithi175 bash[72704]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-mgr-x 2024-09-16T10:35:18.468 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T10:35:18.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:35:18.504 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T10:35:18.504 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T10:35:18.504 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.0 2024-09-16T10:35:18.789 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:18 smithi164 systemd[1]: Stopping Ceph osd.0 for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:18.789 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:35:18 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:35:18.700Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003522181s 2024-09-16T10:35:19.193 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:18 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0[44606]: 2024-09-16T10:35:18.786+0000 7f7997633640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:35:19.193 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:18 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0[44606]: 2024-09-16T10:35:18.786+0000 7f7997633640 -1 osd.0 72 *** Got signal Terminated *** 2024-09-16T10:35:19.193 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:18 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0[44606]: 2024-09-16T10:35:18.786+0000 7f7997633640 -1 osd.0 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T10:35:24.513 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:24 smithi164 podman[87825]: 2024-09-16 10:35:24.223660971 +0000 UTC m=+5.526777908 container died 3a8d46469769040991feb6a8547a4e0e5e8ecf1472620a30f30b69650138caa3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-16T10:35:24.774 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:24 smithi164 podman[87825]: 2024-09-16 10:35:24.694320866 +0000 UTC m=+5.997437792 container cleanup 3a8d46469769040991feb6a8547a4e0e5e8ecf1472620a30f30b69650138caa3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T10:35:24.774 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:24 smithi164 bash[87825]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0 2024-09-16T10:35:25.193 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:24 smithi164 podman[87847]: 2024-09-16 10:35:24.970682594 +0000 UTC m=+0.740407589 container remove 3a8d46469769040991feb6a8547a4e0e5e8ecf1472620a30f30b69650138caa3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T10:35:25.575 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:25 smithi164 podman[87938]: 2024-09-16 10:35:25.178404799 +0000 UTC m=+0.015546859 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T10:35:25.575 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:25 smithi164 podman[87938]: 2024-09-16 10:35:25.304993759 +0000 UTC m=+0.142135813 container create 1077a5ff2b4855ba29523132af6a030a6f2c17ff5eb9d68c420cc48076a23010 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T10:35:25.575 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:25 smithi164 podman[87938]: 2024-09-16 10:35:25.507555563 +0000 UTC m=+0.344697617 container init 1077a5ff2b4855ba29523132af6a030a6f2c17ff5eb9d68c420cc48076a23010 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-16T10:35:25.575 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:25 smithi164 podman[87938]: 2024-09-16 10:35:25.512300731 +0000 UTC m=+0.349442789 container start 1077a5ff2b4855ba29523132af6a030a6f2c17ff5eb9d68c420cc48076a23010 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T10:35:25.575 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:25 smithi164 podman[87938]: 2024-09-16 10:35:25.573820653 +0000 UTC m=+0.410962708 container attach 1077a5ff2b4855ba29523132af6a030a6f2c17ff5eb9d68c420cc48076a23010 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T10:35:25.943 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 16 10:35:25 smithi164 podman[87938]: 2024-09-16 10:35:25.705267967 +0000 UTC m=+0.542410028 container died 1077a5ff2b4855ba29523132af6a030a6f2c17ff5eb9d68c420cc48076a23010 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, GIT_CLEAN=True) 2024-09-16T10:35:26.314 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T10:35:26.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:35:26.355 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T10:35:26.355 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T10:35:26.356 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.1 2024-09-16T10:35:26.694 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:26 smithi164 systemd[1]: Stopping Ceph osd.1 for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:27.193 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1[52041]: 2024-09-16T10:35:26.697+0000 7f7793b0a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:35:27.193 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1[52041]: 2024-09-16T10:35:26.697+0000 7f7793b0a640 -1 osd.1 72 *** Got signal Terminated *** 2024-09-16T10:35:27.193 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:26 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1[52041]: 2024-09-16T10:35:26.697+0000 7f7793b0a640 -1 osd.1 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T10:35:32.374 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:32 smithi164 podman[88066]: 2024-09-16 10:35:32.069160142 +0000 UTC m=+5.484326908 container died ce0f5ec95fdd4ae4c741d5014e9a45543d8129c90df4d7f64dc62d39bcdca867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b) 2024-09-16T10:35:32.693 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:32 smithi164 podman[88066]: 2024-09-16 10:35:32.521762057 +0000 UTC m=+5.936928864 container cleanup ce0f5ec95fdd4ae4c741d5014e9a45543d8129c90df4d7f64dc62d39bcdca867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=reef-e4e227b, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T10:35:32.693 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:32 smithi164 bash[88066]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1 2024-09-16T10:35:33.061 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:32 smithi164 podman[88088]: 2024-09-16 10:35:32.731442207 +0000 UTC m=+0.662514386 container remove ce0f5ec95fdd4ae4c741d5014e9a45543d8129c90df4d7f64dc62d39bcdca867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T10:35:33.336 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:33 smithi164 podman[88182]: 2024-09-16 10:35:32.963116059 +0000 UTC m=+0.016218105 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T10:35:33.336 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:33 smithi164 podman[88182]: 2024-09-16 10:35:33.073837293 +0000 UTC m=+0.126939345 container create a9c9e0eb83c0ea5d686f6ccebe11baf91e1eb3cc32a89bad530bcad2e650b04b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T10:35:33.337 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:33 smithi164 podman[88182]: 2024-09-16 10:35:33.33515311 +0000 UTC m=+0.388255157 container init a9c9e0eb83c0ea5d686f6ccebe11baf91e1eb3cc32a89bad530bcad2e650b04b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T10:35:33.693 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:33 smithi164 podman[88182]: 2024-09-16 10:35:33.340333371 +0000 UTC m=+0.393435420 container start a9c9e0eb83c0ea5d686f6ccebe11baf91e1eb3cc32a89bad530bcad2e650b04b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T10:35:33.694 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:33 smithi164 podman[88182]: 2024-09-16 10:35:33.426688023 +0000 UTC m=+0.479790071 container attach a9c9e0eb83c0ea5d686f6ccebe11baf91e1eb3cc32a89bad530bcad2e650b04b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T10:35:33.694 INFO:journalctl@ceph.osd.1.smithi164.stdout:Sep 16 10:35:33 smithi164 podman[88182]: 2024-09-16 10:35:33.533829379 +0000 UTC m=+0.586931431 container died a9c9e0eb83c0ea5d686f6ccebe11baf91e1eb3cc32a89bad530bcad2e650b04b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T10:35:34.179 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T10:35:34.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:35:34.220 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T10:35:34.221 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T10:35:34.221 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.2 2024-09-16T10:35:34.443 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:34 smithi164 systemd[1]: Stopping Ceph osd.2 for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:34.943 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:34 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2[59757]: 2024-09-16T10:35:34.559+0000 7f2ec9a57640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:35:34.943 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:34 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2[59757]: 2024-09-16T10:35:34.559+0000 7f2ec9a57640 -1 osd.2 72 *** Got signal Terminated *** 2024-09-16T10:35:34.943 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:34 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2[59757]: 2024-09-16T10:35:34.559+0000 7f2ec9a57640 -1 osd.2 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T10:35:40.335 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:40 smithi164 podman[88309]: 2024-09-16 10:35:40.08114953 +0000 UTC m=+5.641753712 container died 07f0b8772cd0d8e7cfb155562b37d2f9e61a8debb49afabdfb7e0e7cf8657111 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T10:35:40.693 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:40 smithi164 podman[88309]: 2024-09-16 10:35:40.516292396 +0000 UTC m=+6.076896567 container cleanup 07f0b8772cd0d8e7cfb155562b37d2f9e61a8debb49afabdfb7e0e7cf8657111 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T10:35:40.693 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:40 smithi164 bash[88309]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2 2024-09-16T10:35:41.123 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:40 smithi164 podman[88329]: 2024-09-16 10:35:40.717711563 +0000 UTC m=+0.634194364 container remove 07f0b8772cd0d8e7cfb155562b37d2f9e61a8debb49afabdfb7e0e7cf8657111 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T10:35:41.443 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:41 smithi164 podman[88424]: 2024-09-16 10:35:41.02527605 +0000 UTC m=+0.015659128 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T10:35:41.443 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:41 smithi164 podman[88424]: 2024-09-16 10:35:41.135154998 +0000 UTC m=+0.125538118 container create 02cafe259569cac609b89bc9905c607836dd398ffafdf75f80ff3d29bd59de20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T10:35:41.443 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:41 smithi164 podman[88424]: 2024-09-16 10:35:41.362464929 +0000 UTC m=+0.352848013 container init 02cafe259569cac609b89bc9905c607836dd398ffafdf75f80ff3d29bd59de20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T10:35:41.443 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:41 smithi164 podman[88424]: 2024-09-16 10:35:41.367558297 +0000 UTC m=+0.357941384 container start 02cafe259569cac609b89bc9905c607836dd398ffafdf75f80ff3d29bd59de20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T10:35:41.849 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:41 smithi164 podman[88424]: 2024-09-16 10:35:41.445586053 +0000 UTC m=+0.435969131 container attach 02cafe259569cac609b89bc9905c607836dd398ffafdf75f80ff3d29bd59de20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T10:35:41.850 INFO:journalctl@ceph.osd.2.smithi164.stdout:Sep 16 10:35:41 smithi164 podman[88424]: 2024-09-16 10:35:41.563901424 +0000 UTC m=+0.554284502 container died 02cafe259569cac609b89bc9905c607836dd398ffafdf75f80ff3d29bd59de20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T10:35:42.281 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T10:35:42.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:35:42.321 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T10:35:42.321 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T10:35:42.321 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.3 2024-09-16T10:35:42.650 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:42 smithi164 systemd[1]: Stopping Ceph osd.3 for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:42.944 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:42 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3[68539]: 2024-09-16T10:35:42.648+0000 7fc2319f4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:35:42.944 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:42 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3[68539]: 2024-09-16T10:35:42.648+0000 7fc2319f4640 -1 osd.3 72 *** Got signal Terminated *** 2024-09-16T10:35:42.944 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:42 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3[68539]: 2024-09-16T10:35:42.648+0000 7fc2319f4640 -1 osd.3 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T10:35:45.193 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:44 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3[68539]: 2024-09-16T10:35:44.925+0000 7fc22e008640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:19.498650+0000 front 2024-09-16T10:35:19.498567+0000 (oldest deadline 2024-09-16T10:35:44.699160+0000) 2024-09-16T10:35:46.193 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:45 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3[68539]: 2024-09-16T10:35:45.896+0000 7fc22e008640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:19.498650+0000 front 2024-09-16T10:35:19.498567+0000 (oldest deadline 2024-09-16T10:35:44.699160+0000) 2024-09-16T10:35:47.193 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:46 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3[68539]: 2024-09-16T10:35:46.895+0000 7fc22e008640 -1 osd.3 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:19.498650+0000 front 2024-09-16T10:35:19.498567+0000 (oldest deadline 2024-09-16T10:35:44.699160+0000) 2024-09-16T10:35:47.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:46 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:46.815+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:47.985 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:47 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:47.823+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:48.443 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:48 smithi164 podman[88550]: 2024-09-16 10:35:48.184554214 +0000 UTC m=+5.640392144 container died 04c78a10d66c74ca13635ebcdbe5438e378da2aa15538827bcefbc42731f9a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, ceph=True) 2024-09-16T10:35:48.735 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:48.606+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:48.808 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:48 smithi164 podman[88550]: 2024-09-16 10:35:48.622606595 +0000 UTC m=+6.078444586 container cleanup 04c78a10d66c74ca13635ebcdbe5438e378da2aa15538827bcefbc42731f9a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T10:35:48.808 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:48 smithi164 bash[88550]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3 2024-09-16T10:35:49.193 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:48 smithi164 podman[88582]: 2024-09-16 10:35:48.807535371 +0000 UTC m=+0.620401580 container remove 04c78a10d66c74ca13635ebcdbe5438e378da2aa15538827bcefbc42731f9a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD) 2024-09-16T10:35:49.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:48.940+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:35:49.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:48 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:48.795+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:49.520 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:49 smithi164 podman[88680]: 2024-09-16 10:35:49.233308261 +0000 UTC m=+0.112210837 container create aff4122b5f6ec70a2efcdd788074435d327176dcc80542ec53c9f6a0e5b2d97c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=reef-e4e227b) 2024-09-16T10:35:49.520 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:49 smithi164 podman[88680]: 2024-09-16 10:35:49.136875485 +0000 UTC m=+0.015778029 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T10:35:49.520 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:49 smithi164 podman[88680]: 2024-09-16 10:35:49.435406265 +0000 UTC m=+0.314308798 container init aff4122b5f6ec70a2efcdd788074435d327176dcc80542ec53c9f6a0e5b2d97c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2) 2024-09-16T10:35:49.520 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:49 smithi164 podman[88680]: 2024-09-16 10:35:49.440116084 +0000 UTC m=+0.319018621 container start aff4122b5f6ec70a2efcdd788074435d327176dcc80542ec53c9f6a0e5b2d97c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T10:35:49.520 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:49 smithi164 podman[88680]: 2024-09-16 10:35:49.51865305 +0000 UTC m=+0.397555593 container attach aff4122b5f6ec70a2efcdd788074435d327176dcc80542ec53c9f6a0e5b2d97c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T10:35:49.912 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:49.650+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:49.913 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:49.832+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:49.943 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 16 10:35:49 smithi164 podman[88680]: 2024-09-16 10:35:49.636993312 +0000 UTC m=+0.515895855 container died aff4122b5f6ec70a2efcdd788074435d327176dcc80542ec53c9f6a0e5b2d97c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T10:35:50.235 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:49.909+0000 7f19d8ac4640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.107837+0000 front 2024-09-16T10:35:23.108108+0000 (oldest deadline 2024-09-16T10:35:49.008020+0000) 2024-09-16T10:35:50.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:49 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:49.922+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:35:50.305 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T10:35:50.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:35:50.344 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T10:35:50.344 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T10:35:50.344 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.4 2024-09-16T10:35:50.648 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:50 smithi175 systemd[1]: Stopping Ceph osd.4 for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:50.648 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:50.646+0000 7f19dc4b0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:35:50.648 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:50.646+0000 7f19dc4b0640 -1 osd.4 72 *** Got signal Terminated *** 2024-09-16T10:35:50.649 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:50.646+0000 7f19dc4b0640 -1 osd.4 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T10:35:50.924 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:50.693+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:50.925 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:50.803+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:50.925 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:50.875+0000 7f19d8ac4640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.107837+0000 front 2024-09-16T10:35:23.108108+0000 (oldest deadline 2024-09-16T10:35:49.008020+0000) 2024-09-16T10:35:51.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:50 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:50.923+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:35:51.959 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:51 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:51.653+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:51.959 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:51 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:51.858+0000 7f19d8ac4640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.107837+0000 front 2024-09-16T10:35:23.108108+0000 (oldest deadline 2024-09-16T10:35:49.008020+0000) 2024-09-16T10:35:51.959 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:51 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:51.810+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:52.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:51 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:51.957+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:35:52.913 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:52.647+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:52.913 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:52.775+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:52.913 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:52.815+0000 7f19d8ac4640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.107837+0000 front 2024-09-16T10:35:23.108108+0000 (oldest deadline 2024-09-16T10:35:49.008020+0000) 2024-09-16T10:35:52.913 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:52.815+0000 7f19d8ac4640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.008552+0000 front 2024-09-16T10:35:29.008644+0000 (oldest deadline 2024-09-16T10:35:52.508508+0000) 2024-09-16T10:35:53.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:52 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:52.911+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:35:53.915 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:53 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:53.735+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:53.915 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:53 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:53.790+0000 7f19d8ac4640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.107837+0000 front 2024-09-16T10:35:23.108108+0000 (oldest deadline 2024-09-16T10:35:49.008020+0000) 2024-09-16T10:35:53.915 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:53 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:53.790+0000 7f19d8ac4640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.008552+0000 front 2024-09-16T10:35:29.008644+0000 (oldest deadline 2024-09-16T10:35:52.508508+0000) 2024-09-16T10:35:53.915 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:53 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:53.614+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:53.915 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:53 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:53.614+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:35:54.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:53 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:53.913+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:35:54.897 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:54 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:54.821+0000 7f19d8ac4640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.107837+0000 front 2024-09-16T10:35:23.108108+0000 (oldest deadline 2024-09-16T10:35:49.008020+0000) 2024-09-16T10:35:54.897 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:54 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4[35624]: 2024-09-16T10:35:54.821+0000 7f19d8ac4640 -1 osd.4 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.008552+0000 front 2024-09-16T10:35:29.008644+0000 (oldest deadline 2024-09-16T10:35:52.508508+0000) 2024-09-16T10:35:54.898 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:54 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:54.632+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:54.898 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:54 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:54.632+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:35:54.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:54 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:54.756+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:54.898 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:54 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:54.756+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:35:55.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:54 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:54.895+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:35:55.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:54 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:54.895+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:28.319366+0000 front 2024-09-16T10:35:28.319352+0000 (oldest deadline 2024-09-16T10:35:54.219152+0000) 2024-09-16T10:35:55.889 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:55 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:55.600+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:55.889 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:55 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:55.600+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:35:55.889 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:55 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:55.729+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:55.889 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:55 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:55.729+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:35:56.235 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:56 smithi175 podman[72851]: 2024-09-16 10:35:56.046775097 +0000 UTC m=+5.507507876 container died 079392e29ec992ae333487149fce9f563298cbd39b09ea38b7ce628d165ecc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T10:35:56.236 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:55 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:55.886+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:35:56.236 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:55 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:55.886+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:28.319366+0000 front 2024-09-16T10:35:28.319352+0000 (oldest deadline 2024-09-16T10:35:54.219152+0000) 2024-09-16T10:35:56.605 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:56 smithi175 podman[72851]: 2024-09-16 10:35:56.411659057 +0000 UTC m=+5.872391836 container cleanup 079392e29ec992ae333487149fce9f563298cbd39b09ea38b7ce628d165ecc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-16T10:35:56.605 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:56 smithi175 bash[72851]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4 2024-09-16T10:35:56.605 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:56 smithi175 podman[72871]: 2024-09-16 10:35:56.579422583 +0000 UTC m=+0.535372888 container remove 079392e29ec992ae333487149fce9f563298cbd39b09ea38b7ce628d165ecc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T10:35:56.931 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:56 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:56.602+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:56.932 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:56 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:56.602+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:35:56.932 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:56 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:56.717+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:56.932 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:56 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:56.717+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:35:57.235 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:57 smithi175 podman[72965]: 2024-09-16 10:35:56.923312114 +0000 UTC m=+0.016288126 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T10:35:57.235 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:57 smithi175 podman[72965]: 2024-09-16 10:35:57.038775634 +0000 UTC m=+0.131751646 container create 8fc3c7e4de6670b16ddba29b15f2ecac6ce541e49792ebec00e88d7add465279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T10:35:57.236 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:56 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:56.929+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:35:57.236 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:56 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:56.929+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:28.319366+0000 front 2024-09-16T10:35:28.319352+0000 (oldest deadline 2024-09-16T10:35:54.219152+0000) 2024-09-16T10:35:57.575 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:57 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:57.573+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:57.576 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:57 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:57.573+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:35:57.576 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:57 smithi175 podman[72965]: 2024-09-16 10:35:57.316392144 +0000 UTC m=+0.409368161 container init 8fc3c7e4de6670b16ddba29b15f2ecac6ce541e49792ebec00e88d7add465279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T10:35:57.576 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:57 smithi175 podman[72965]: 2024-09-16 10:35:57.321770013 +0000 UTC m=+0.414746028 container start 8fc3c7e4de6670b16ddba29b15f2ecac6ce541e49792ebec00e88d7add465279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T10:35:57.576 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:57 smithi175 podman[72965]: 2024-09-16 10:35:57.382891208 +0000 UTC m=+0.475867240 container attach 8fc3c7e4de6670b16ddba29b15f2ecac6ce541e49792ebec00e88d7add465279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T10:35:57.576 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 16 10:35:57 smithi175 podman[72965]: 2024-09-16 10:35:57.525803611 +0000 UTC m=+0.618779625 container died 8fc3c7e4de6670b16ddba29b15f2ecac6ce541e49792ebec00e88d7add465279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b) 2024-09-16T10:35:57.838 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:57 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:57.742+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:57.838 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:57 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:57.742+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:35:58.105 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:57 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:57.893+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:35:58.105 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:57 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:57.893+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:28.319366+0000 front 2024-09-16T10:35:28.319352+0000 (oldest deadline 2024-09-16T10:35:54.219152+0000) 2024-09-16T10:35:58.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T10:35:58.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:35:58.174 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T10:35:58.174 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T10:35:58.174 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.5 2024-09-16T10:35:58.485 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:58 smithi175 systemd[1]: Stopping Ceph osd.5 for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:35:58.756 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:58 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:58.605+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:58.756 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:58 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:58.605+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:35:58.756 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:58 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:58.533+0000 7f8a0a753640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:35:58.756 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:58 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:58.533+0000 7f8a0a753640 -1 osd.5 72 *** Got signal Terminated *** 2024-09-16T10:35:58.756 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:58 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:58.533+0000 7f8a0a753640 -1 osd.5 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T10:35:59.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:58 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:58.854+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:35:59.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:58 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:58.854+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:28.319366+0000 front 2024-09-16T10:35:28.319352+0000 (oldest deadline 2024-09-16T10:35:54.219152+0000) 2024-09-16T10:35:59.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:58 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:58.754+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:59.236 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:58 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:58.754+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:35:59.854 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:59 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:59.563+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:35:59.854 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:35:59 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:35:59.563+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:35:59.854 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:59 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:59.781+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:35:59.854 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:35:59 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:35:59.781+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:00.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:59 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:59.851+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:36:00.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:35:59 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:35:59.852+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:28.319366+0000 front 2024-09-16T10:35:28.319352+0000 (oldest deadline 2024-09-16T10:35:54.219152+0000) 2024-09-16T10:36:00.828 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:00 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:00.547+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:36:00.829 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:00 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:00.547+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:36:01.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:00 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:00.826+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:01.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:00 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:00.826+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:01.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:00 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:00.826+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:01.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:00 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:36:00.887+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:36:01.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:00 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:36:00.887+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:28.319366+0000 front 2024-09-16T10:35:28.319352+0000 (oldest deadline 2024-09-16T10:35:54.219152+0000) 2024-09-16T10:36:01.923 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:01 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:01.593+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:36:01.923 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:01 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:01.593+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:36:01.923 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:01 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:01.831+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:01.924 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:01 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:01.831+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:01.924 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:01 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:01.831+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:02.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:01 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:36:01.921+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:36:02.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:01 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:36:01.921+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:28.319366+0000 front 2024-09-16T10:35:28.319352+0000 (oldest deadline 2024-09-16T10:35:54.219152+0000) 2024-09-16T10:36:02.933 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:02 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:02.623+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:36:02.933 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:02 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:02.623+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:36:02.934 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:02 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:02.816+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:02.934 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:02 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:02.816+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:02.934 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:02 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:02.816+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:03.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:02 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:36:02.931+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.018886+0000 front 2024-09-16T10:35:23.018932+0000 (oldest deadline 2024-09-16T10:35:48.318764+0000) 2024-09-16T10:36:03.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:02 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5[43001]: 2024-09-16T10:36:02.931+0000 7f8a06d67640 -1 osd.5 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:28.319366+0000 front 2024-09-16T10:35:28.319352+0000 (oldest deadline 2024-09-16T10:35:54.219152+0000) 2024-09-16T10:36:03.986 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:03 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:03.615+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:36:03.986 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:03 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:03.615+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:36:03.986 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:03 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:03.816+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:03.986 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:03 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:03.816+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:03.986 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:03 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:03.816+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:04.275 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:03 smithi175 podman[73091]: 2024-09-16 10:36:03.998500775 +0000 UTC m=+5.595783314 container died 4484c1b3834106c40b61335e49aa785caba44fc22b95a2d19059fdb1b6bf1a3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T10:36:04.625 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:04 smithi175 podman[73091]: 2024-09-16 10:36:04.364890016 +0000 UTC m=+5.962172554 container cleanup 4484c1b3834106c40b61335e49aa785caba44fc22b95a2d19059fdb1b6bf1a3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T10:36:04.625 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:04 smithi175 bash[73091]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5 2024-09-16T10:36:04.981 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:04 smithi175 podman[73111]: 2024-09-16 10:36:04.624409859 +0000 UTC m=+0.619088528 container remove 4484c1b3834106c40b61335e49aa785caba44fc22b95a2d19059fdb1b6bf1a3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=reef-e4e227b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T10:36:04.981 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:04 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:04.806+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:04.981 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:04 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:04.806+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:04.981 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:04 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:04.806+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:04.982 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:04 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:04.662+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:36:04.982 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:04 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:04.662+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:36:04.982 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:04 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:04.662+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:38.151386+0000 front 2024-09-16T10:35:38.151405+0000 (oldest deadline 2024-09-16T10:36:04.051336+0000) 2024-09-16T10:36:05.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:04 smithi175 podman[73208]: 2024-09-16 10:36:04.882689485 +0000 UTC m=+0.015963600 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T10:36:05.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:04 smithi175 podman[73208]: 2024-09-16 10:36:04.983378957 +0000 UTC m=+0.116653066 container create 78c3c8ac2854738ffa8a873758efc587180249ecab4ea0a6ec2ff36c0e283855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=reef-e4e227b, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T10:36:05.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:05 smithi175 podman[73208]: 2024-09-16 10:36:05.169487917 +0000 UTC m=+0.302762027 container init 78c3c8ac2854738ffa8a873758efc587180249ecab4ea0a6ec2ff36c0e283855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T10:36:05.235 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:05 smithi175 podman[73208]: 2024-09-16 10:36:05.174856005 +0000 UTC m=+0.308130116 container start 78c3c8ac2854738ffa8a873758efc587180249ecab4ea0a6ec2ff36c0e283855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T10:36:05.616 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:05 smithi175 podman[73208]: 2024-09-16 10:36:05.252626074 +0000 UTC m=+0.385900189 container attach 78c3c8ac2854738ffa8a873758efc587180249ecab4ea0a6ec2ff36c0e283855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T10:36:05.616 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 16 10:36:05 smithi175 podman[73208]: 2024-09-16 10:36:05.367028448 +0000 UTC m=+0.500302559 container died 78c3c8ac2854738ffa8a873758efc587180249ecab4ea0a6ec2ff36c0e283855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b) 2024-09-16T10:36:05.890 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:05 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:05.614+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:36:05.890 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:05 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:05.614+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:36:05.890 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:05 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:05.614+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:38.151386+0000 front 2024-09-16T10:35:38.151405+0000 (oldest deadline 2024-09-16T10:36:04.051336+0000) 2024-09-16T10:36:05.891 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:05 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:05.834+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:05.891 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:05 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:05.834+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:05.891 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:05 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:05.834+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:05.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T10:36:05.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:36:05.948 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T10:36:05.948 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T10:36:05.948 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.6 2024-09-16T10:36:06.235 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:06 smithi175 systemd[1]: Stopping Ceph osd.6 for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:36:06.595 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:06 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:06.277+0000 7f7cbc35c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:36:06.595 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:06 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:06.277+0000 7f7cbc35c640 -1 osd.6 72 *** Got signal Terminated *** 2024-09-16T10:36:06.595 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:06 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:06.277+0000 7f7cbc35c640 -1 osd.6 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T10:36:06.595 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:06 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:06.593+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:36:06.595 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:06 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:06.593+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:36:06.595 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:06 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:06.593+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:38.151386+0000 front 2024-09-16T10:35:38.151405+0000 (oldest deadline 2024-09-16T10:36:04.051336+0000) 2024-09-16T10:36:07.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:06 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:06.867+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:07.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:06 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:06.867+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:07.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:06 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:06.867+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:07.905 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:07 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:07.553+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:36:07.905 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:07 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:07.553+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:36:07.905 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:07 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:07.553+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:38.151386+0000 front 2024-09-16T10:35:38.151405+0000 (oldest deadline 2024-09-16T10:36:04.051336+0000) 2024-09-16T10:36:08.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:07 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:07.903+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:08.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:07 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:07.903+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:08.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:07 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:07.903+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:08.900 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:08 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:08.598+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:36:08.901 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:08 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:08.598+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:36:08.901 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:08 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:08.598+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:38.151386+0000 front 2024-09-16T10:35:38.151405+0000 (oldest deadline 2024-09-16T10:36:04.051336+0000) 2024-09-16T10:36:09.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:08 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:08.898+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:09.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:08 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:08.898+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:09.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:08 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:08.898+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:09.932 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:09 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:09.574+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:36:09.932 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:09 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:09.574+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:36:09.932 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:09 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:09.574+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:38.151386+0000 front 2024-09-16T10:35:38.151405+0000 (oldest deadline 2024-09-16T10:36:04.051336+0000) 2024-09-16T10:36:09.932 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:09 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:09.574+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:44.051756+0000 front 2024-09-16T10:35:44.051768+0000 (oldest deadline 2024-09-16T10:36:08.751699+0000) 2024-09-16T10:36:10.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:09 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:09.930+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:10.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:09 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:09.930+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:10.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:09 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:09.930+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:10.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:09 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:09.930+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:45.267514+0000 front 2024-09-16T10:35:45.267449+0000 (oldest deadline 2024-09-16T10:36:09.367365+0000) 2024-09-16T10:36:10.900 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:10 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:10.564+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:22.448914+0000 front 2024-09-16T10:35:22.448807+0000 (oldest deadline 2024-09-16T10:35:47.649431+0000) 2024-09-16T10:36:10.900 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:10 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:10.564+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.950440+0000 front 2024-09-16T10:35:29.950453+0000 (oldest deadline 2024-09-16T10:35:53.450292+0000) 2024-09-16T10:36:10.900 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:10 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:10.564+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:38.151386+0000 front 2024-09-16T10:35:38.151405+0000 (oldest deadline 2024-09-16T10:36:04.051336+0000) 2024-09-16T10:36:10.900 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:10 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6[51473]: 2024-09-16T10:36:10.564+0000 7f7cb4968640 -1 osd.6 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:44.051756+0000 front 2024-09-16T10:35:44.051768+0000 (oldest deadline 2024-09-16T10:36:08.751699+0000) 2024-09-16T10:36:11.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:10 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:10.898+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:11.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:10 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:10.898+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:11.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:10 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:10.898+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:11.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:10 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:10.898+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:45.267514+0000 front 2024-09-16T10:35:45.267449+0000 (oldest deadline 2024-09-16T10:36:09.367365+0000) 2024-09-16T10:36:11.961 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:11 smithi175 podman[73334]: 2024-09-16 10:36:11.683865572 +0000 UTC m=+5.518199911 container died db31cf7d699e59e1985cb5571a9c87f9720c8cd15c7ab999cb29a98ee776fd64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T10:36:11.962 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:11 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:11.880+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:11.962 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:11 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:11.880+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:11.962 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:11 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:11.880+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:11.962 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:11 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:11.880+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:45.267514+0000 front 2024-09-16T10:35:45.267449+0000 (oldest deadline 2024-09-16T10:36:09.367365+0000) 2024-09-16T10:36:12.235 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:12 smithi175 podman[73334]: 2024-09-16 10:36:12.059279592 +0000 UTC m=+5.893613919 container cleanup db31cf7d699e59e1985cb5571a9c87f9720c8cd15c7ab999cb29a98ee776fd64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=reef-e4e227b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T10:36:12.235 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:12 smithi175 bash[73334]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6 2024-09-16T10:36:12.617 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:12 smithi175 podman[73354]: 2024-09-16 10:36:12.302263838 +0000 UTC m=+0.620675233 container remove db31cf7d699e59e1985cb5571a9c87f9720c8cd15c7ab999cb29a98ee776fd64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T10:36:12.956 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:12 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:12.858+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:12.956 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:12 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:12.858+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:12.956 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:12 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:12.858+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:12.956 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:12 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:12.858+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:45.267514+0000 front 2024-09-16T10:35:45.267449+0000 (oldest deadline 2024-09-16T10:36:09.367365+0000) 2024-09-16T10:36:12.956 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:12 smithi175 podman[73450]: 2024-09-16 10:36:12.519857964 +0000 UTC m=+0.015968274 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T10:36:12.956 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:12 smithi175 podman[73450]: 2024-09-16 10:36:12.62801136 +0000 UTC m=+0.124121662 container create 26257c3da04c0b71220a919e3d71e5b9df9a7cadf2e51571b4fe8564665bfcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T10:36:12.957 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:12 smithi175 podman[73450]: 2024-09-16 10:36:12.846958468 +0000 UTC m=+0.343068768 container init 26257c3da04c0b71220a919e3d71e5b9df9a7cadf2e51571b4fe8564665bfcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T10:36:12.957 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:12 smithi175 podman[73450]: 2024-09-16 10:36:12.852500981 +0000 UTC m=+0.348611292 container start 26257c3da04c0b71220a919e3d71e5b9df9a7cadf2e51571b4fe8564665bfcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T10:36:12.957 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:12 smithi175 podman[73450]: 2024-09-16 10:36:12.95520667 +0000 UTC m=+0.451316984 container attach 26257c3da04c0b71220a919e3d71e5b9df9a7cadf2e51571b4fe8564665bfcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=reef-e4e227b) 2024-09-16T10:36:13.235 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:13 smithi175 podman[73450]: 2024-09-16 10:36:13.044180633 +0000 UTC m=+0.540290946 container died 26257c3da04c0b71220a919e3d71e5b9df9a7cadf2e51571b4fe8564665bfcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, GIT_CLEAN=True, ceph=True) 2024-09-16T10:36:13.578 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 16 10:36:13 smithi175 podman[73494]: 2024-09-16 10:36:13.575562557 +0000 UTC m=+0.525449033 container remove 26257c3da04c0b71220a919e3d71e5b9df9a7cadf2e51571b4fe8564665bfcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T10:36:13.603 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T10:36:13.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:36:13.644 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T10:36:13.645 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T10:36:13.645 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.7 2024-09-16T10:36:13.855 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:13 smithi175 systemd[1]: Stopping Ceph osd.7 for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:36:14.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:13 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:13.853+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:14.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:13 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:13.853+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:14.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:13 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:13.853+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:14.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:13 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:13.853+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:45.267514+0000 front 2024-09-16T10:35:45.267449+0000 (oldest deadline 2024-09-16T10:36:09.367365+0000) 2024-09-16T10:36:14.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:13 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:13.955+0000 7fbac108e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:36:14.236 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:13 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:13.955+0000 7fbac108e640 -1 osd.7 72 *** Got signal Terminated *** 2024-09-16T10:36:14.236 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:13 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:13.955+0000 7fbac108e640 -1 osd.7 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T10:36:15.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:14.873+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:15.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:14.873+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:15.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:14.873+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:15.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:14 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:14.873+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:45.267514+0000 front 2024-09-16T10:35:45.267449+0000 (oldest deadline 2024-09-16T10:36:09.367365+0000) 2024-09-16T10:36:16.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:15.876+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:16.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:15.876+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:16.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:15.876+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:16.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:15 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:15.876+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:45.267514+0000 front 2024-09-16T10:35:45.267449+0000 (oldest deadline 2024-09-16T10:36:09.367365+0000) 2024-09-16T10:36:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:16.003Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.164:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.164:8765: connect: connection refused" 2024-09-16T10:36:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:16.003Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.164:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.164:8765: connect: connection refused" 2024-09-16T10:36:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:16.007Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.164:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.164:8765: connect: connection refused" 2024-09-16T10:36:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:16.007Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.164:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.164:8765: connect: connection refused" 2024-09-16T10:36:16.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:16.007Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.164:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.164:8765: connect: connection refused" 2024-09-16T10:36:17.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:16.912+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:17.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:16.912+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:17.236 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:16.912+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:17.236 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:16.912+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:45.267514+0000 front 2024-09-16T10:35:45.267449+0000 (oldest deadline 2024-09-16T10:36:09.367365+0000) 2024-09-16T10:36:17.236 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:16 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:16.912+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.175:6804 osd.4 since back 2024-09-16T10:35:50.468349+0000 front 2024-09-16T10:35:50.468395+0000 (oldest deadline 2024-09-16T10:36:16.368456+0000) 2024-09-16T10:36:18.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:17 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:17.932+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:18.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:17 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:17.932+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:18.236 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:17 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:17.932+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:18.236 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:17 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:17.932+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:45.267514+0000 front 2024-09-16T10:35:45.267449+0000 (oldest deadline 2024-09-16T10:36:09.367365+0000) 2024-09-16T10:36:18.236 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:17 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:17.932+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.175:6804 osd.4 since back 2024-09-16T10:35:50.468349+0000 front 2024-09-16T10:35:50.468395+0000 (oldest deadline 2024-09-16T10:36:16.368456+0000) 2024-09-16T10:36:19.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:18 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:18.904+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-16T10:35:23.463570+0000 front 2024-09-16T10:35:23.463415+0000 (oldest deadline 2024-09-16T10:35:46.363424+0000) 2024-09-16T10:36:19.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:18 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:18.904+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6814 osd.1 since back 2024-09-16T10:35:29.764885+0000 front 2024-09-16T10:35:29.764923+0000 (oldest deadline 2024-09-16T10:35:53.864771+0000) 2024-09-16T10:36:19.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:18 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:18.904+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6822 osd.2 since back 2024-09-16T10:35:36.066040+0000 front 2024-09-16T10:35:36.066051+0000 (oldest deadline 2024-09-16T10:36:00.765919+0000) 2024-09-16T10:36:19.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:18 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:18.904+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.164:6830 osd.3 since back 2024-09-16T10:35:45.267514+0000 front 2024-09-16T10:35:45.267449+0000 (oldest deadline 2024-09-16T10:36:09.367365+0000) 2024-09-16T10:36:19.235 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:18 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7[59497]: 2024-09-16T10:36:18.904+0000 7fbab969a640 -1 osd.7 72 heartbeat_check: no reply from 172.21.15.175:6804 osd.4 since back 2024-09-16T10:35:50.468349+0000 front 2024-09-16T10:35:50.468395+0000 (oldest deadline 2024-09-16T10:36:16.368456+0000) 2024-09-16T10:36:19.655 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:19 smithi175 podman[73580]: 2024-09-16 10:36:19.356110318 +0000 UTC m=+5.495792753 container died cdfe990d2f05a642fae4cd91aede0107fe61ebb0aebf9629a96e994e6346f58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T10:36:19.985 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:19 smithi175 podman[73580]: 2024-09-16 10:36:19.752831069 +0000 UTC m=+5.892513500 container cleanup cdfe990d2f05a642fae4cd91aede0107fe61ebb0aebf9629a96e994e6346f58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T10:36:19.985 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:19 smithi175 bash[73580]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7 2024-09-16T10:36:20.313 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:20 smithi175 podman[73612]: 2024-09-16 10:36:20.003743744 +0000 UTC m=+0.644745292 container remove cdfe990d2f05a642fae4cd91aede0107fe61ebb0aebf9629a96e994e6346f58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=reef-e4e227b) 2024-09-16T10:36:20.591 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:20 smithi175 podman[73706]: 2024-09-16 10:36:20.214752747 +0000 UTC m=+0.016233976 image pull 67ddd33fa72f26a51c969a9043e7a3ddf784506b3d1949ad815ae4dd791beec1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322 2024-09-16T10:36:20.591 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:20 smithi175 podman[73706]: 2024-09-16 10:36:20.312799861 +0000 UTC m=+0.114281096 container create 07e670472a7a510e825b7097728e3dbcfb4db47796be1cf9f522b4746c33aa27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T10:36:20.591 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:20 smithi175 podman[73706]: 2024-09-16 10:36:20.506930045 +0000 UTC m=+0.308411274 container init 07e670472a7a510e825b7097728e3dbcfb4db47796be1cf9f522b4746c33aa27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-e4e227b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T10:36:20.591 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:20 smithi175 podman[73706]: 2024-09-16 10:36:20.511557386 +0000 UTC m=+0.313038616 container start 07e670472a7a510e825b7097728e3dbcfb4db47796be1cf9f522b4746c33aa27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=reef-e4e227b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T10:36:20.592 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:20 smithi175 podman[73706]: 2024-09-16 10:36:20.590229524 +0000 UTC m=+0.391710772 container attach 07e670472a7a510e825b7097728e3dbcfb4db47796be1cf9f522b4746c33aa27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T10:36:20.985 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 16 10:36:20 smithi175 podman[73706]: 2024-09-16 10:36:20.702335967 +0000 UTC m=+0.503817197 container died 07e670472a7a510e825b7097728e3dbcfb4db47796be1cf9f522b4746c33aa27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T10:36:21.338 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T10:36:21.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:36:21.376 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T10:36:21.377 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-16T10:36:21.377 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-16T10:36:21.664 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:36:21 smithi164 systemd[1]: Stopping Ceph rgw.foo.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:36:21.943 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:36:21 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-rgw-foo-a[76166]: 2024-09-16T10:36:21.660+0000 7f4b25c83640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T10:36:21.943 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:36:21 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-rgw-foo-a[76166]: 2024-09-16T10:36:21.660+0000 7f4b294ee880 -1 shutting down 2024-09-16T10:36:31.944 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:36:31 smithi164 bash[88806]: time="2024-09-16T10:36:31Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-3d5087c4-7416-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-16T10:36:31.944 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:36:31 smithi164 podman[88806]: 2024-09-16 10:36:31.682960405 +0000 UTC m=+10.116007978 container died 79aded71f0dc1566d24204977fd27c5ced2a1cd1a0a09604ef9950aafe84ad0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T10:36:31.945 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:36:31 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:36:31.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi164.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.164:8443: connect: connection refused" 2024-09-16T10:36:31.945 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:36:31 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:36:31.627Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-16T10:36:32.413 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:36:32 smithi164 podman[88806]: 2024-09-16 10:36:32.176531073 +0000 UTC m=+10.609578686 container cleanup 79aded71f0dc1566d24204977fd27c5ced2a1cd1a0a09604ef9950aafe84ad0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T10:36:32.413 INFO:journalctl@ceph.rgw.foo.a.smithi164.stdout:Sep 16 10:36:32 smithi164 bash[88806]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-16T10:36:32.542 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-16T10:36:32.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:36:32.579 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-16T10:36:32.579 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-16T10:36:32.579 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-3d5087c4-7416-11ef-bceb-c7b262605968@prometheus.a 2024-09-16T10:36:32.873 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 systemd[1]: Stopping Ceph prometheus.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:36:33.235 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.871Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T10:36:33.235 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.871Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-16T10:36:33.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.871Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-16T10:36:33.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.871Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T10:36:33.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.871Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-16T10:36:33.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.871Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-16T10:36:33.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.872Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T10:36:33.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.872Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-16T10:36:33.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.872Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-16T10:36:33.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.872Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T10:36:33.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.872Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-16T10:36:33.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a[72601]: ts=2024-09-16T10:36:32.872Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-16T10:36:33.236 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:32 smithi175 podman[73832]: 2024-09-16 10:36:32.882624293 +0000 UTC m=+0.116955549 container died 4175c42b42c5064f10a4ebc7f65a32317f98c51c5f02c96e3a76a1092a84e620 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T10:36:33.719 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:33 smithi175 podman[73832]: 2024-09-16 10:36:33.311659605 +0000 UTC m=+0.545990861 container cleanup 4175c42b42c5064f10a4ebc7f65a32317f98c51c5f02c96e3a76a1092a84e620 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T10:36:33.720 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:33 smithi175 bash[73832]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T10:36:33.720 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 16 10:36:33 smithi175 podman[73849]: 2024-09-16 10:36:33.504619978 +0000 UTC m=+0.620106696 container remove 4175c42b42c5064f10a4ebc7f65a32317f98c51c5f02c96e3a76a1092a84e620 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T10:36:33.748 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3d5087c4-7416-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-16T10:36:33.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T10:36:33.784 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-16T10:36:33.784 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T10:36:33.967 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:36:35.693 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:36:35 smithi164 systemd[1]: Stopping Ceph alertmanager.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:36:36.065 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:36:35 smithi164 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a[87142]: ts=2024-09-16T10:36:35.807Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-16T10:36:36.065 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:36:35 smithi164 podman[89276]: 2024-09-16 10:36:35.82766209 +0000 UTC m=+0.135106180 container died 762aba524889ef76181c2316094d44a2f7668794211967c70d5accd3a5f8491b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:36:36.349 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:36:36 smithi164 podman[89276]: 2024-09-16 10:36:36.163165303 +0000 UTC m=+0.470609381 container cleanup 762aba524889ef76181c2316094d44a2f7668794211967c70d5accd3a5f8491b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:36:36.349 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:36:36 smithi164 bash[89276]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a 2024-09-16T10:36:36.693 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:36:36 smithi164 podman[89295]: 2024-09-16 10:36:36.347948189 +0000 UTC m=+0.521662566 container remove 762aba524889ef76181c2316094d44a2f7668794211967c70d5accd3a5f8491b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T10:36:36.693 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:36:36 smithi164 podman[89295]: 2024-09-16 10:36:36.432252313 +0000 UTC m=+0.605966690 volume remove 7fb02c3c65b62ea9ad02c214a056456a7f9aea46fb24851dd0d3ddde1004d254 2024-09-16T10:36:37.165 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:36:36 smithi164 systemd[1]: Stopping Ceph node-exporter.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:36:37.165 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:36:36 smithi164 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-16T10:36:37.165 INFO:journalctl@ceph.alertmanager.a.smithi164.stdout:Sep 16 10:36:36 smithi164 systemd[1]: Stopped Ceph alertmanager.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:36:37.443 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:36:37 smithi164 podman[89407]: 2024-09-16 10:36:37.164125265 +0000 UTC m=+0.106165344 container died 707b167eba288a4cfcff655dbf5f9a5c76d9bad49540f48c826bbe03b902866f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T10:36:37.872 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:36:37 smithi164 podman[89407]: 2024-09-16 10:36:37.694111183 +0000 UTC m=+0.636151265 container cleanup 707b167eba288a4cfcff655dbf5f9a5c76d9bad49540f48c826bbe03b902866f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T10:36:37.872 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:36:37 smithi164 bash[89407]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a 2024-09-16T10:36:37.872 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:36:37 smithi164 podman[89424]: 2024-09-16 10:36:37.870032542 +0000 UTC m=+0.709936817 container remove 707b167eba288a4cfcff655dbf5f9a5c76d9bad49540f48c826bbe03b902866f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T10:36:38.124 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:36:37 smithi164 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-16T10:36:38.124 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:36:38 smithi164 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-16T10:36:38.124 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:36:38 smithi164 systemd[1]: Stopped Ceph node-exporter.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:36:38.125 INFO:journalctl@ceph.node-exporter.a.smithi164.stdout:Sep 16 10:36:38 smithi164 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 1.900s CPU time. 2024-09-16T10:36:38.754 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T10:36:38.906 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:36:40.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:40 smithi175 systemd[1]: Stopping Ceph iscsi.iscsi.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:36:40.735 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:40 smithi175 ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a[67093]: debug Shutdown received 2024-09-16T10:36:50.646 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:50 smithi175 bash[74213]: time="2024-09-16T10:36:50Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-16T10:36:50.646 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:50 smithi175 podman[74213]: 2024-09-16 10:36:50.396719951 +0000 UTC m=+10.124863715 container died 1e9a78c1fad707560118af6790e7a697b74511fcc50d2f9bd2568ba15cd46e69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2) 2024-09-16T10:36:50.985 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:50 smithi175 podman[74213]: 2024-09-16 10:36:50.830987387 +0000 UTC m=+10.559131158 container cleanup 1e9a78c1fad707560118af6790e7a697b74511fcc50d2f9bd2568ba15cd46e69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-e4e227b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T10:36:50.985 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:50 smithi175 bash[74213]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a 2024-09-16T10:36:51.348 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:51 smithi175 podman[74236]: 2024-09-16 10:36:51.019874163 +0000 UTC m=+0.623637732 container remove 1e9a78c1fad707560118af6790e7a697b74511fcc50d2f9bd2568ba15cd46e69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=reef-e4e227b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T10:36:51.348 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:51 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-16T10:36:51.711 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:51 smithi175 podman[74286]: 2024-09-16 10:36:51.347562373 +0000 UTC m=+0.107024431 container died 380f4010ab5ddcd12ca85e990dba0ace65e109da093cda481c01a1f4d84644a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=reef-e4e227b) 2024-09-16T10:36:51.985 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:51 smithi175 podman[74286]: 2024-09-16 10:36:51.801798649 +0000 UTC m=+0.561260736 container cleanup 380f4010ab5ddcd12ca85e990dba0ace65e109da093cda481c01a1f4d84644a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T10:36:51.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:51 smithi175 bash[74286]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-16T10:36:51.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:51 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-16T10:36:51.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:51 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 66947 (conmon) remains running after unit stopped. 2024-09-16T10:36:51.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:51 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Unit process 74303 (podman) remains running after unit stopped. 2024-09-16T10:36:51.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:51 smithi175 systemd[1]: Stopped Ceph iscsi.iscsi.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:36:51.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:51 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Consumed 2.265s CPU time. 2024-09-16T10:36:52.485 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:36:52 smithi175 systemd[1]: Stopping Ceph grafana.a for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:36:52.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 16 10:36:52 smithi175 podman[74303]: 2024-09-16 10:36:52.095077797 +0000 UTC m=+0.751514505 container remove 380f4010ab5ddcd12ca85e990dba0ace65e109da093cda481c01a1f4d84644a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:94ec087b0651344ea44bf4ca432cec3d5bcba32599243d0588e250fe952c8322, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-iscsi-iscsi-a-tcmu, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=reef-e4e227b, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T10:36:52.750 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:36:52 smithi175 podman[74429]: 2024-09-16 10:36:52.485794766 +0000 UTC m=+0.106100397 container died bada0a3c81e0a09572a613b9082cad98bb49b2c77db0369fce2ff09d6f09a2e3 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, distribution-scope=public, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, name=ubi8, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git) 2024-09-16T10:36:53.213 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:36:52 smithi175 podman[74429]: 2024-09-16 10:36:52.805988248 +0000 UTC m=+0.426293876 container cleanup bada0a3c81e0a09572a613b9082cad98bb49b2c77db0369fce2ff09d6f09a2e3 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a, name=ubi8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vendor=Red Hat, Inc., architecture=x86_64, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, io.buildah.version=1.32.0, io.openshift.tags=base rhel8, description=Ceph Grafana Container, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git, distribution-scope=public, maintainer=Paul Cuzner , com.redhat.component=ubi8-container, build-date=2022-04-29T11:48:11.055897, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration) 2024-09-16T10:36:53.214 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:36:52 smithi175 bash[74429]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a 2024-09-16T10:36:53.214 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:36:52 smithi175 podman[74454]: 2024-09-16 10:36:52.990495667 +0000 UTC m=+0.506611403 container remove bada0a3c81e0a09572a613b9082cad98bb49b2c77db0369fce2ff09d6f09a2e3 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-grafana-a, build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, description=Ceph Grafana Container, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, vendor=Red Hat, Inc., io.openshift.tags=base rhel8, vcs-type=git, architecture=x86_64) 2024-09-16T10:36:53.214 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:36:52 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-16T10:36:53.485 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:36:53 smithi175 systemd[1]: Stopping Ceph node-exporter.b for 3d5087c4-7416-11ef-bceb-c7b262605968... 2024-09-16T10:36:53.485 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:36:53 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-16T10:36:53.485 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:36:53 smithi175 systemd[1]: Stopped Ceph grafana.a for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:36:53.485 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 16 10:36:53 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@grafana.a.service: Consumed 3.462s CPU time. 2024-09-16T10:36:53.985 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:36:53 smithi175 podman[74571]: 2024-09-16 10:36:53.681317782 +0000 UTC m=+0.118925145 container died 3e319ee4b38bfa7928c662e056580c05e87c3f208d4b4dfb0bee0c82015232bc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T10:36:54.398 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:36:54 smithi175 podman[74571]: 2024-09-16 10:36:54.010182064 +0000 UTC m=+0.447789424 container cleanup 3e319ee4b38bfa7928c662e056580c05e87c3f208d4b4dfb0bee0c82015232bc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T10:36:54.398 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:36:54 smithi175 bash[74571]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b 2024-09-16T10:36:54.398 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:36:54 smithi175 podman[74588]: 2024-09-16 10:36:54.18614092 +0000 UTC m=+0.508531081 container remove 3e319ee4b38bfa7928c662e056580c05e87c3f208d4b4dfb0bee0c82015232bc (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3d5087c4-7416-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T10:36:54.398 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:36:54 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-16T10:36:54.735 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:36:54 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-16T10:36:54.735 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:36:54 smithi175 systemd[1]: Stopped Ceph node-exporter.b for 3d5087c4-7416-11ef-bceb-c7b262605968. 2024-09-16T10:36:54.735 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 16 10:36:54 smithi175 systemd[1]: ceph-3d5087c4-7416-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 1.881s CPU time. 2024-09-16T10:36:55.091 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T10:36:55.124 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T10:36:55.151 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T10:36:55.153 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906653/remote/smithi164/crash 2024-09-16T10:36:55.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/crash -- . 2024-09-16T10:36:55.203 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T10:36:55.203 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2024-09-16T10:36:55.205 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906653/remote/smithi175/crash 2024-09-16T10:36:55.206 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/crash -- . 2024-09-16T10:36:55.252 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T10:36:55.252 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2024-09-16T10:36:55.253 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T10:36:55.253 DEBUG:teuthology.orchestra.run.smithi164:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v CEPHADM_STRAY_DAEMON | egrep -v CEPHADM_FAILED_DAEMON | egrep -v CEPHADM_AGENT_DOWN | head -n 1 2024-09-16T10:36:55.283 INFO:teuthology.orchestra.run.smithi164.stderr:grep: /var/log/ceph/3d5087c4-7416-11ef-bceb-c7b262605968/ceph.log: No such file or directory 2024-09-16T10:36:55.285 INFO:tasks.cephadm:Compressing logs... 2024-09-16T10:36:55.285 DEBUG:teuthology.orchestra.run.smithi164:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T10:36:55.327 DEBUG:teuthology.orchestra.run.smithi175:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T10:36:55.349 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T10:36:55.351 INFO:teuthology.orchestra.run.smithi175.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T10:36:56.809 INFO:tasks.cephadm:Archiving logs... 2024-09-16T10:36:56.810 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906653/remote/smithi164/log 2024-09-16T10:36:56.811 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T10:36:57.059 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906653/remote/smithi175/log 2024-09-16T10:36:57.059 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T10:36:57.201 INFO:tasks.cephadm:Removing cluster... 2024-09-16T10:36:57.202 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 --force 2024-09-16T10:36:57.340 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:36:57.466 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3d5087c4-7416-11ef-bceb-c7b262605968 --force 2024-09-16T10:36:57.618 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: 3d5087c4-7416-11ef-bceb-c7b262605968 2024-09-16T10:36:57.745 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T10:36:57.746 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T10:36:57.762 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T10:36:57.777 INFO:tasks.cephadm:Teardown complete 2024-09-16T10:36:57.777 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-09-16T10:36:57.788 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi164:/dev/vg_nvme/lv_1... 2024-09-16T10:36:57.788 DEBUG:teuthology.orchestra.run.smithi164:> sudo nvme disconnect -n lv_1 2024-09-16T10:36:57.946 INFO:teuthology.orchestra.run.smithi164.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-16T10:36:57.948 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi164:/dev/vg_nvme/lv_2... 2024-09-16T10:36:57.949 DEBUG:teuthology.orchestra.run.smithi164:> sudo nvme disconnect -n lv_2 2024-09-16T10:36:58.096 INFO:teuthology.orchestra.run.smithi164.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-16T10:36:58.098 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi164:/dev/vg_nvme/lv_3... 2024-09-16T10:36:58.098 DEBUG:teuthology.orchestra.run.smithi164:> sudo nvme disconnect -n lv_3 2024-09-16T10:36:58.252 INFO:teuthology.orchestra.run.smithi164.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-16T10:36:58.254 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi164:/dev/vg_nvme/lv_4... 2024-09-16T10:36:58.254 DEBUG:teuthology.orchestra.run.smithi164:> sudo nvme disconnect -n lv_4 2024-09-16T10:36:58.408 INFO:teuthology.orchestra.run.smithi164.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-16T10:36:58.410 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T10:36:58.410 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/scratch_devs 2024-09-16T10:36:58.435 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi175:/dev/vg_nvme/lv_1... 2024-09-16T10:36:58.435 DEBUG:teuthology.orchestra.run.smithi175:> sudo nvme disconnect -n lv_1 2024-09-16T10:36:58.592 INFO:teuthology.orchestra.run.smithi175.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-16T10:36:58.593 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi175:/dev/vg_nvme/lv_2... 2024-09-16T10:36:58.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo nvme disconnect -n lv_2 2024-09-16T10:36:58.758 INFO:teuthology.orchestra.run.smithi175.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-16T10:36:58.761 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi175:/dev/vg_nvme/lv_3... 2024-09-16T10:36:58.761 DEBUG:teuthology.orchestra.run.smithi175:> sudo nvme disconnect -n lv_3 2024-09-16T10:36:58.920 INFO:teuthology.orchestra.run.smithi175.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-16T10:36:58.922 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi175:/dev/vg_nvme/lv_4... 2024-09-16T10:36:58.922 DEBUG:teuthology.orchestra.run.smithi175:> sudo nvme disconnect -n lv_4 2024-09-16T10:36:59.072 INFO:teuthology.orchestra.run.smithi175.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-16T10:36:59.074 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-16T10:36:59.074 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/scratch_devs 2024-09-16T10:36:59.104 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T10:36:59.112 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T10:36:59.112 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T10:36:59.115 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T10:36:59.130 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-16T10:36:59.134 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T10:36:59.135 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-16T10:36:59.135 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 -606us[ -588us] +/- 54ms 2024-09-16T10:36:59.135 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 +1300us[+1300us] +/- 60ms 2024-09-16T10:36:59.135 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -773us[ -756us] +/- 79ms 2024-09-16T10:36:59.135 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:36:59.160 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-09-16T10:36:59.165 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T10:36:59.165 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-09-16T10:36:59.165 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 -607us[ -593us] +/- 54ms 2024-09-16T10:36:59.165 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 +1301us[+1301us] +/- 60ms 2024-09-16T10:36:59.165 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 -766us[ -766us] +/- 79ms 2024-09-16T10:36:59.165 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:36:59.166 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T10:36:59.175 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T10:36:59.175 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T10:36:59.189 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T10:36:59.272 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T10:36:59.349 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T10:36:59.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:36:59.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T10:36:59.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:36:59.410 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T10:36:59.419 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T10:36:59.432 INFO:teuthology.task.internal:Duration was 821.151237 seconds 2024-09-16T10:36:59.433 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T10:36:59.447 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T10:36:59.447 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T10:36:59.450 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T10:36:59.488 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T10:36:59.490 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T10:36:59.981 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T10:36:59.981 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T10:36:59.981 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T10:37:00.044 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-09-16T10:37:00.044 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T10:37:00.071 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T10:37:00.071 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T10:37:00.087 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T10:37:00.142 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T10:37:00.142 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T10:37:00.156 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T10:37:00.421 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T10:37:00.436 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T10:37:00.436 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T10:37:00.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T10:37:00.486 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T10:37:00.501 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T10:37:00.504 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T10:37:00.526 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-09-16T10:37:00.549 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-09-16T10:37:00.579 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T10:37:00.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:37:00.608 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T10:37:00.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:37:00.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T10:37:00.650 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T10:37:00.651 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906653/remote/smithi164 2024-09-16T10:37:00.651 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T10:37:00.689 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906653/remote/smithi175 2024-09-16T10:37:00.690 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T10:37:00.725 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T10:37:00.725 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T10:37:00.728 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T10:37:00.774 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T10:37:00.787 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T10:37:00.788 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T10:37:00.801 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T10:37:00.802 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T10:37:00.804 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T10:37:00.817 INFO:teuthology.orchestra.run.smithi164.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 10:37 /home/ubuntu/cephtest 2024-09-16T10:37:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 10:37 /home/ubuntu/cephtest 2024-09-16T10:37:00.830 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T10:37:00.841 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T10:37:00.875 INFO:teuthology.run:Summary data: description: rados/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} duration: 821.1512367725372 owner: scheduled_teuthology@teuthology success: true 2024-09-16T10:37:00.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T10:37:00.957 INFO:teuthology.run:pass